From 0ce0107f4f16cfbbfbff7d9a71079f935b96b942 Mon Sep 17 00:00:00 2001 From: Joan Leon Date: Thu, 27 Aug 2026 18:37:32 +0200 Subject: [PATCH] feat(performance): deepen backend and database substance #358 broadened the performance-optimization description to cover "frontend, backend, queries, and databases", but the body's backend coverage was a single N+1 example, against the full Core Web Vitals and art-direction treatment on the frontend side. Routing a "slow query" or "connection pool" prompt here landed on a snippet. Adds three measure-first patterns inline, each with a "when NOT to": - Query plans: read EXPLAIN ANALYZE before indexing; composite column order (equality first, then range/sort); the four cases where an index does not help; revert an index that did not change the plan. - Connection pool exhaustion: the distinctive symptom, one pool per process, why a bigger pool relocates the queue instead of removing it, and the proxy answer for serverless. - Caching: layer choice (in-process, shared, edge), key design as a correctness concern, one invalidation strategy rather than an accidental mix, and stampede protection. Deeper material goes to references/performance-checklist.md to keep SKILL.md readable: read/write patterns, negative caching, request coalescing, and query-plan, index, pooling and cache checklists. Closes #378 Claude-Session: https://claude.ai/code/session_01TuWNaEV9RFXpTxqeBRTRg8 --- references/performance-checklist.md | 83 +++++++++++++++++++++++ skills/performance-optimization/SKILL.md | 85 ++++++++++++++++++++++++ 2 files changed, 168 insertions(+) diff --git a/references/performance-checklist.md b/references/performance-checklist.md index 86a41496..b771c794 100644 --- a/references/performance-checklist.md +++ b/references/performance-checklist.md @@ -8,6 +8,7 @@ Quick reference checklist for web application performance. Use alongside the `pe - [TTFB Diagnosis](#ttfb-diagnosis) - [Frontend Checklist](#frontend-checklist) - [Backend Checklist](#backend-checklist) +- [Caching Strategies](#caching-strategies) - [Measurement Commands](#measurement-commands) - [Common Anti-Patterns](#common-anti-patterns) @@ -92,6 +93,30 @@ When TTFB is slow (> 800ms), check each component in DevTools Network waterfall: - [ ] Connection pooling configured - [ ] Slow query logging enabled +#### Query plans +- [ ] `EXPLAIN ANALYZE` captured **before** the fix, not just after — it is the baseline +- [ ] `Seq Scan` on a large table understood: index missing, unusable, or genuinely not worth it +- [ ] Estimated vs actual `rows=` within an order of magnitude (if not, refresh statistics before touching indexes) +- [ ] No `Sort` node that a composite index could absorb +- [ ] Plan re-checked after the change — an index that did not change the plan gets reverted + +#### Index strategy +- [ ] Composite index column order is equality first, then range/sort +- [ ] Index covers the query shape (filter + sort), not just one column in isolation +- [ ] Covering index considered for hot read paths (index-only scan avoids the heap fetch) +- [ ] Not indexing low-selectivity columns (one value dominating the table) +- [ ] Expression index used where the query applies a function (`lower(email)`) +- [ ] Full-text or trigram index used for leading-wildcard search, not a B-tree +- [ ] Write cost measured on write-heavy tables (every index taxes every `INSERT`/`UPDATE`) +- [ ] Unused and duplicate indexes dropped (they cost writes and buy nothing) + +#### Connection pooling +- [ ] One pool per process, not per request or per module +- [ ] `instances × pool max` stays under the database's `max_connections` +- [ ] `connectionTimeoutMillis` set so exhaustion fails fast instead of queueing forever +- [ ] Exhaustion diagnosed before resizing: find what holds connections (long transactions, missing `await`, leaked clients) +- [ ] Serverless / autoscaling fronted by a multiplexing proxy (pgbouncer, RDS Proxy) rather than a larger pool + ### API - [ ] Response times < 200ms (p95) - [ ] No synchronous heavy computation in request handlers @@ -105,6 +130,58 @@ When TTFB is slow (> 800ms), check each component in DevTools Network waterfall: - [ ] Horizontal scaling configured (if needed) - [ ] Health check endpoint for load balancer +## Caching Strategies + +The decision material (which layer, which invalidation strategy, what never to cache) lives in the `performance-optimization` skill. This section covers the read/write patterns and the checklist. + +### Read and write patterns + +| Pattern | How it works | Use when | Watch out for | +|---|---|---|---| +| **Cache-aside** (lazy) | App checks cache, on miss reads origin and populates | Default choice; read-heavy, tolerant of a cold first hit | Every miss hits the origin, so it needs stampede protection | +| **Read-through** | Cache layer itself loads on miss | You want the load path in one place, not at every call site | Hides origin latency; a slow origin looks like a slow cache | +| **Write-through** | Write goes to cache and origin together, synchronously | Reads must never see a stale value after a write | Adds cache latency to every write | +| **Write-behind** (write-back) | Write hits cache, origin updated asynchronously | Write-heavy, and the origin is the bottleneck | Data loss window if the cache dies before the flush. Needs durability you can defend | + +### Negative caching + +Cache the *absence* of a result too. A key that misses on every lookup (a nonexistent user ID probed in a loop, a 404 asset) sends every request to the origin, which is a cache that only protects the happy path. + +- Store an explicit "not found" sentinel with a **shorter** TTL than positive entries +- Keep the negative TTL short enough that a newly created record appears promptly +- Never let an origin *error* become a negative cache entry, or one failing minute becomes many + +### Request coalescing (stampede protection) + +One recompute, N waiters. Prevents a hot key's expiry from delivering the full concurrent load to the origin: + +```typescript +const inFlight = new Map>(); + +function loadOnce(key: string, fetcher: () => Promise): Promise { + const existing = inFlight.get(key); + if (existing) return existing; + const p = fetcher().finally(() => inFlight.delete(key)); + inFlight.set(key, p); + return p; +} +``` + +For a shared cache, the same idea needs a distributed lock, or `stale-while-revalidate` so waiters serve the stale value instead of blocking. + +### Cache checklist +- [ ] The cached call was measured as expensive first (caching a fast call adds a hop and buys nothing) +- [ ] Read/write ratio justifies the cache (re-read far more often than written) +- [ ] Cache key includes every input the response varies on: tenant, viewer, locale, permissions, feature flags +- [ ] No per-user data cached under a key that does not identify the user +- [ ] One invalidation strategy chosen (TTL, event/tag, or versioned keys), not an accidental mix +- [ ] Acceptable staleness window written down, not implied by whatever TTL was typed +- [ ] Stampede protection on hot keys (coalescing, lock, or `stale-while-revalidate`) +- [ ] Negative results cached with a shorter TTL; origin errors never cached +- [ ] Eviction policy and memory ceiling set (an unbounded cache is a memory leak) +- [ ] Hit rate monitored — a cache nobody measures is an assumption, and a low hit rate is pure overhead +- [ ] Nothing cached whose staleness is a correctness bug (balances, permissions, inventory at checkout) + ## Measurement Commands ### INP field data and DevTools workflow @@ -146,6 +223,12 @@ onINP(({ value, attribution }) => { | N+1 queries | Linear DB load growth | Use joins, includes, or batch loading | | Unbounded queries | Memory exhaustion, timeouts | Always paginate, add LIMIT | | Missing indexes | Slow reads as data grows | Add indexes for filtered/sorted columns | +| Indexing without reading the plan | Write cost paid, read gain unproven | `EXPLAIN ANALYZE` before and after; revert if the plan is unchanged | +| Redundant / unused indexes | Every write pays for them | Audit usage stats, drop what nothing reads | +| Connection pool per request | Exhausts `max_connections` under load | One pool per process; proxy for serverless | +| Cache key missing the viewer | One user's data served to another | Key on tenant, viewer, locale, permissions | +| Unbounded cache | Memory leak wearing an optimization's clothing | Set eviction policy and a memory ceiling | +| Cache stampede on a hot key | Origin takes full concurrent load at expiry | Coalesce misses, or `stale-while-revalidate` | | Layout thrashing | Jank, dropped frames | Batch DOM reads, then batch writes | | Unoptimized images | Slow LCP, wasted bandwidth | Use WebP, responsive sizes, lazy load | | Large bundles | Slow Time to Interactive | Code split, tree shake, audit deps | diff --git a/skills/performance-optimization/SKILL.md b/skills/performance-optimization/SKILL.md index a4e968e5..8becaa48 100644 --- a/skills/performance-optimization/SKILL.md +++ b/skills/performance-optimization/SKILL.md @@ -149,6 +149,58 @@ const tasks = await db.tasks.findMany({ }); ``` +#### Queries That Ignore Their Index + +"Add an index" is the guess. The query plan is the measurement: + +```sql +EXPLAIN ANALYZE +SELECT id, title FROM tasks +WHERE owner_id = 42 ORDER BY created_at DESC LIMIT 20; +``` + +Three things in the output decide the fix: + +| What you see | What it means | +|---|---| +| `Seq Scan` on a large table where you expected an index | No usable index for this predicate | +| Estimated `rows=` off from actual by an order of magnitude | Stale statistics; the planner is choosing on bad information | +| A `Sort` node above the scan | The index covers the filter but not the `ORDER BY` | + +Index for the **shape of the query**, not the column in isolation. In a composite index, equality columns come first, then the range or sort column: + +```sql +CREATE INDEX idx_tasks_owner_created ON tasks (owner_id, created_at DESC); +``` + +**When an index will not help:** + +| Situation | Why | +|---|---| +| Low selectivity (a `status` column that is 95% one value) | A sequential scan is genuinely cheaper; the planner will ignore the index | +| Leading wildcard (`LIKE '%term'`) | A B-tree cannot seek without a prefix; needs trigram or full-text | +| Function on the column (`WHERE lower(email) = ?`) | The plain column index is unusable; index the expression instead | +| Write-heavy table | Every index is a tax on every `INSERT`/`UPDATE`; measure the write cost, not just the read gain | + +Re-run `EXPLAIN ANALYZE` after. An index that did not change the plan is a revert (Step 4), and it is not free: it still costs on every write. + +#### Connection Pool Exhaustion + +The signature is distinctive: **every** endpoint slows at once, the slow time is spent waiting for a connection rather than executing, and the database reports mostly idle sessions. + +```typescript +// BAD: a pool per request or per module — under serverless this multiplies +// by instance count and exhausts the database's connection limit +// GOOD: one pool per process, sized against the database's ceiling +const pool = new Pool({ + max: 10, // instances × max must stay under max_connections + idleTimeoutMillis: 30_000, + connectionTimeoutMillis: 5_000, // fail fast instead of queueing forever +}); +``` + +**Bigger is not faster.** A pool larger than what the database can execute concurrently just relocates the queue from your app to the database, where it is harder to see. When instance count is unbounded (serverless, autoscaling), a proxy that multiplexes connections (pgbouncer, RDS Proxy) is the fix, not a higher `max`. + #### Missing Image Optimization (Frontend) ```html @@ -264,6 +316,16 @@ function App() { #### Missing Caching (Backend) +Cache what is expensive to produce and read far more often than it changes. Caching a query that was already fast adds a network hop, a staleness bug, and an eviction policy to maintain, in exchange for nothing. + +**Pick the layer deliberately:** + +| Layer | Visible to | Use when | Cost | +|---|---|---|---| +| In-process (`Map`, LRU) | One instance | Small, hot, per-instance staleness is acceptable | Each instance drifts independently; invalidation reaches only one | +| Shared (Redis, Memcached) | All instances | Instances must agree, or the value is expensive to recompute | A network hop, and another service to run and monitor | +| CDN / edge | Everyone, per URL | Responses are public and identical for a given key | Invalidation is the hard part; assume you cannot recall a bad response quickly | + ```typescript // Cache frequently-read, rarely-changed data const CACHE_TTL = 5 * 60 * 1000; // 5 minutes @@ -289,6 +351,20 @@ app.use('/static', express.static('public', { res.set('Cache-Control', 'public, max-age=300'); // 5 minutes ``` +**Key design decides correctness.** Every input that changes the response belongs in the key: tenant, locale, permissions, feature flags. A key that omits the viewer is how one user's data gets served to another, and that ships as a performance win. + +**Choose one invalidation strategy, not three:** + +| Strategy | Trade-off | +|---|---| +| TTL | Simplest. You accept staleness up to the TTL, so state the acceptable window explicitly | +| Event or tag based | Fresh on write, but writers now have to know the cache topology | +| Versioned keys (`user:42:profile:v7`) | Never invalidate, just stop reading old keys. Costs memory until eviction | + +**Guard against the stampede.** A hot key expires, every concurrent request misses together, and the origin takes the full load at once, which is how a cache turns into an outage instead of preventing one. Serve stale while a single request recomputes (`stale-while-revalidate`), or coalesce concurrent misses behind one in-flight promise so N waiters cause one recompute. + +**Do not cache:** anything whose staleness is a correctness bug (balances, permissions, inventory at checkout), or per-user data under a key that does not identify the user. See `../../references/performance-checklist.md` for request coalescing, write strategies, negative caching, and the cache checklist. + ### Step 4: Verify (Keep or Revert) A fix is a hypothesis until you re-measure. This step decides whether it survives. @@ -361,6 +437,9 @@ For detailed performance checklists, optimization commands, and anti-pattern ref | "This optimization is obvious" | If you didn't measure, you don't know. Profile first. | | "Users won't notice 100ms" | Research shows 100ms delays impact conversion rates. Users notice more than you think. | | "The framework handles performance" | Frameworks prevent some issues but can't fix N+1 queries or oversized bundles. | +| "The query is slow, add an index" | Read the plan first. The index may already exist and be unusable, and every index taxes writes forever. | +| "Just cache it" | Caching an already-cheap call buys nothing and adds a staleness bug. Cache what is expensive *and* re-read far more than written. | +| "Raise the pool size, we're running out of connections" | A pool bigger than the database can serve moves the queue somewhere less visible. Find what holds connections. | | "It didn't help much, but it doesn't hurt" | Neutral changes are a revert. You pay maintenance on them forever and got nothing back. | | "We already wrote it, may as well keep it" | Sunk cost. The measurement doesn't care how long the change took to write. | | "The improvement is obvious, no need to re-measure" | Then re-measuring is cheap and proves it. Unmeasured wins are how neutral complexity lands. | @@ -369,6 +448,10 @@ For detailed performance checklists, optimization commands, and anti-pattern ref - Optimization without profiling data to justify it - N+1 query patterns in data fetching +- An index added without a query plan before and after to justify it +- A cache key that omits an input the response depends on (tenant, locale, viewer) +- A cache with no stated staleness window and no invalidation strategy +- Connection pool size raised in response to exhaustion, without finding what holds connections - List endpoints without pagination - Images without dimensions, lazy loading, or responsive sizes - Bundle size growing without review @@ -392,5 +475,7 @@ After any performance-related change: - [ ] Core Web Vitals are within "Good" thresholds - [ ] Bundle size hasn't increased significantly - [ ] No N+1 queries in new data fetching code +- [ ] Any new index is justified by a query plan before and after, and its write cost was considered +- [ ] Any new cache states what it keys on and how it goes stale - [ ] Performance budget passes in CI (if configured) - [ ] Existing tests still pass (optimization didn't break behavior)