Thundering Herd
Also known as:
Cache Stampede
Dogpile Effect
Definition
The thundering herd problem happens when many requests hit the same backend resource at the same instant, typically right after a popular cache key expires. Every request sees the miss, every one queries the database together, and the sudden spike can take the database down. Fixes spread or collapse that simultaneous load rather than letting it all land at once.
Popular reads
View All
Payment System Design: Ledger, Idempotency, and Settlement
Jul 18, 2026
The Complete HTMX Guide: From Zero to Production
Dec 22, 2025
How Google manages billions of lines of code in one monorepo
Sep 04, 2026
Cursor Skills: How to Create and Use Agent Skills
Jun 23, 2026
Transactional Outbox Pattern: Never Lose an Event Again
Apr 07, 2026
Flash Sale System Design: Architecture, Scale, and Oversell
May 16, 2026
Key Takeaways
- It is the spike that follows a synchronised event, classically a hot cache key expiring for everyone at the same moment.
- Adding random jitter to TTLs stops many keys from expiring together.
- Request coalescing (singleflight) lets one request rebuild the value while the rest wait for it.
- Distributed locks with stale-data fallback or probabilistic early recomputation also tame the herd.
How It Works
- A widely-read cache entry expires, so the next wave of requests all miss at once.
- Without protection, every missed request independently hits the database to recompute the value.
- The database sees a sudden burst far above its steady load and can stall or crash.
- Mitigations either spread the misses out (jitter) or collapse them into a single recompute (coalescing, locks).
Where It Is Used
- Go’s singleflight package coalesces duplicate in-flight calls for the same key.
- CDNs use request collapsing so one origin fetch serves many waiting clients.
- Pairing a circuit breaker and rate limiting protects the backend during a stampede.