Performance engineering

if you made something 2x faster, you might have done something smart
if you made something 100x faster, you definitely just stopped doing something stupid
— @rygorous

Two layers: a methodology discipline (don’t guess — observe, characterize, then act) and a priorities ladder (the fastest work is the work you never do). Both ends are captured in the vault:

The priorities ladder — Hanson & Crain’s mantras

Craig Hanson and Pat Crain’s performance mantras (via Brendan Gregg’s methodology page, capture: clippings/brendangregg-com-methodology-html.md), in priority order:

1. Don’t do it — the fastest code is no code; elimination beats optimization
2. Do it, but don’t do it again — cache (memoize results, not recompute)
3. Do it less — reduce frequency, batch, filter early
4. Do it later — async/defer: move work off the critical path
5. Do it when they’re not looking — schedule for idle windows
6. Do it concurrently — parallelize what remains
7. Do it cheaper — only now algorithm/hardware-level optimization

The whole ladder is ordered by leverage per unit of effort — which is why “don’t do it” is step one and micro-optimization is step seven. @BenjDicken’s version: “Doing nothing is wayyyyy faster than doing something” (clippings/x-2105707994549985458.md). @rygorous sharper still: “if you made something 2x faster, you might have done something smart; if you made something 100x faster, you definitely just stopped doing something stupid” (clippings/x-1271296834439282690.md) — big wins are almost always elimination, not optimization.

Ladder applied for real — PlanetScale’s Neki (PostgreSQL sharding proxy; capture: clippings/planetscale-com-blog-designing-neki-for-performance.md): a router between client and Postgres can’t remove its own network hop, so the perf work becomes what the router can avoid doing — don’t decode results you don’t need to understand; if you need one column, let the other ten stay as raw protocol bytes. Lazy + composable decoding (“walking the messages ≠ constructing rows; finding rows ≠ decoding columns”). The comparison point: Vitess decodes everything up front into protobuf rows — the “sensible default” Neki deliberately escapes.

The methodology discipline — Gregg’s catalog

From the same methodology page: performance work without a method degenerates into the anti-methods (Blame-Someone-Else, Streetlight, Drunk Man, Random Change, Passive Benchmarking). The real ones, in rough order of when to reach for them:

• Problem Statement — has this system ever performed well? what changed recently? can the degradation be expressed as latency/ops?
• Workload Characterization — who/why/what/how-much is the load (PID, code path, IOPS, time series)
• USE Method — per resource: Utilization, Saturation, Errors (finds bottlenecks)
• RED Method — per service: Rate, Errors, Duration (Tom Wilkie; the services complement to USE)
• Off-CPU Analysis — where threads wait, not where they burn CPU (latency lives here)
• Active Benchmarking — analyze the benchmark WHILE it runs to find the limiting factor, instead of publishing result slides (passive benchmarking’s antidote)
• Time Division / Drill-Down — split latency into components until the origin is found; quantify the payoff before fixing

Plus the stopping rule: “try several methodologies on a theory before moving on” — a week per theory, not six months on one issue. And Gregg’s Benchmarking Checklist: why not double? did it break limits? does it reproduce? does it matter? did it even happen?

SQL indexes — the elimination ladder applied to reads

Indexing is performance engineering where the elimination step has a data structure: an index lets the database not read the rows it doesn’t need. The same ladder applies verbatim — before optimizing a query (step 6), ask whether a B-tree lookup can delete the full-table scan entirely (step 2).

The core mental model (from Use The Index Luke): an index is a B-tree of doubly-linked leaf nodes — logarithmic search plus linked-list range walks. Slowness comes from two ingredients: tree traversal (depth) and table access (each leaf hit fetching the row from the heap). Most tuning is minimizing the second: concatenated keys ordered by equality-then-range columns, index-only scans (covering indexes — the table access disappears), pipelined ORDER BY/GROUP BY (the index provides the sort for free), and the seek method for pagination (offset re-scans everything it skips; seek starts at the key).

The classic self-inflicted wounds: functions on the indexed column (UPPER(email) = can’t use the plain index — function-based index or immutable-case column), type-mixing in comparisons (numeric strings, date obfuscation — “databases don’t solve equations”), and over-indexing — every index taxes every INSERT/UPDATE/DELETE (DML maintenance is the write-side cost the read-side win hides). Partial indexes (WHERE status = 'pending') are the index world’s own minimalism: index only the rows that matter.

Neki’s lazy-decode lesson reads differently with this background: covering indexes and Neki’s raw-bytes pass-through are the same move — don’t materialize what the question doesn’t need.

Resources: Use The Index Luke — Markus Winand’s free SQL indexing & tuning e-book (captured → clippings/use-the-index-luke-com.md with the full chapter map): index anatomy → WHERE clause operators → testing/scalability → joins (N+1, hash vs nested loops) → clustering/covering → sorting/grouping pipelining → partial results/pagination → DML costs. Database-agnostic (Oracle/SQL Server/MySQL/Postgres). Companion site: modern-sql.com.

Related: Kurs administracji PostgreSQL (EXPLAIN-driven tuning lessons) · Mail servers/OSS DB work · the DML-cost point returns in Working with legacy code — “the schema you can’t safely change”

The memory and cache canon (two primers, both captured)

• Gallery of Processor Cache Effects (Igor Ostrovsky, 2010 — clippings/igoro-com-archive-gallery-of-processor-cache-effects.md): the classic empirical walk — Loop 1 vs Loop 2 (i++ vs i += 16) runs at nearly the same speed because both touch every cache line; then the effects ladder: line-effects, hardware prefetching, cache associativity, and the “effects on program performance” conclusions. The 2^n step experiment making loops faster at strides 15→16 is the single most convincing cache demo ever written.
• Memory Allocation Strategies (Ginger Bill / odin author — clippings/gingerbill-org-series-memory-allocation-strategies.md, 6-part index): thinking about memory → linear/arena, stack, pool, free-list, buddy allocators. The allocator is a data-structure choice; the series is the standard modern reference for hand-rolled allocators (the lane parakeet.cpp and any systems work eventually needs).

Both land on the same rung of the ladder below: memory layout is the “stop doing something stupid” tier — a 100x from cache-friendliness beats any micro-optimization. @vivekgalatage surfaced the pair together (“a must read if you are into memory mgmt”).

Where it fits here

• Postgres angle: Neki’s lazy-decode lesson generalizes to any proxy/router layer (and the Kurs administracji PostgreSQL tuning lessons are step-6/7 territory; elimination is step 1)
• LLM-era restatement: as compute gets cheap, “don’t do it” and “do it later” dominate — aggressive caching of agent/inference results before any model-level optimization (Inference engines)
• Benchmarking discipline pairs with the Agent tests no-tautology rule: a benchmark that can’t fail teaches nothing

Related: Diff tools (flame graphs are diffable artifacts too) · Golden repo (benchmark gates in CI)

hyperfine 2.0: the benchmark harness standard (2026-10-08)

charliermarsh (173❤; 4-tweet thread) — hyperfine 2.0 is out (sharkdp/hyperfine, 29k★ Rust; tweet + OSS note captured): the headline feature is alternative performance metrics — peak memory usage, CPU cycles, and more — alongside wall time; on Linux also cache hits, cache misses, branch misses (perf counters). The release cements hyperfine as the command-line benchmarking harness: statistical runs (warmup, outlier detection, min/mean/max/σ), shell-command fixtures, export to JSON/CSV/Markdown, --parameter-scan for matrix benchmarks. USER TESTIMONIAL, verbatim: “I used hyperfine for performance testing and love it.” For this note’s measure-first doctrine, hyperfine is the tool half: any A/B claim (“is this faster?”) starts as hyperfine 'cmd-a' 'cmd-b' — and v2’s memory/cycle metrics mean the comparison no longer stops at wall time (the “faster but hungrier” regression now shows up in the same table).