campaign-sender · visual documentation · 1 of 2

Architecture

How 1,000,000 personalized emails flow through one small pipeline — and why a crash can't cause duplicates.

System flow

One streaming pass: the recipient file is never held in memory. Batches of 1,000 pass two gates — a semaphore bounding work in flight (our memory and sockets) and a token bucket bounding request starts/sec (the provider's quota) — into the transport seam; every outcome has exactly one path out.

recipients.csv 1M rows · ~40 MB stream reader line-by-line batcher 1,000 / batch two gates semaphore · 20 in flight token bucket · --rate req/s Transport dry-run · ESP bulk API backpressure — a full semaphore pauses the file read ✓ success checkpoint ← fsync ↻ transient (429 / 5xx) backoff + jitter · ≤ 5 attempts re-enqueue batch ✕ permanent reject rejected.log — never retried campaign.checkpoint append-only · one line per batch Memory stays flat: O(concurrency × batch) ≈ 20,000 rows in RAM, whether the list is 1M or 100M.
success → durable record transient → retried permanent → logged, skipped
The pipeline knows no provider — Transport.sendBatch() is the single seam. Full prose: docs/02-architecture.md.

Crash & resume — the checkpoint at work

Measured demo: the process is hard-killed (exit 137, no cleanup) after ~400k sends. The rerun reads the checkpoint, skips the 400 completed batches, and finishes the remaining 600. Final checkpoint: 1,000 unique indices, zero duplicates.

Run 1 pnpm send --crash-after 400000 400 batches sent + checkpointed ✕ hard crash (kill −9 equivalent) at ~40% campaign.checkpoint 400 fsync'd batch indices survive Run 2 pnpm send · 4.6s 400 batches skipped — read, not resent 600 batches sent this run verified: tail -n +2 campaign.checkpoint | sort -n | uniq -d | wc -l → 0
sent in this run skipped via checkpoint crash point
The checkpoint is written after the provider accepts a batch and fsync'd before anything else observes success — so the worst-case duplicate window is one batch, never "everything since start". Full prose: docs/03-reliability.md.

Every failure has one exit

The retry unit is the batch, matching the transport's atomicity. Classification happens at the transport boundary — only it knows what a provider's status codes mean.

sendBatch(batch) ✓ 2xx — provider accepted markDone(index) → fsync rejects within batch → rejected.log ↻ TransientError 429 · 5xx · network drop nothing was sent — safe to retry ✕ non-retryable 4xx malformed request · bad auth every batch would fail identically 5xx: attempt < 5 · 429: within 60s budget exhausted 429: Retry-After (60s time budget) 5xx: 250ms·2ⁿ × jitter, ≤5 attempts stop loudly — checkpoint keeps all completed work rerun resumes from exactly here: failure path = crash path
Silently dropping batches would be the only failure worse than duplicates — so malformed requests stop the campaign instead of being swallowed.