# When does a write-behind cache TTL below the flush interval silently revert acknowledged writes? **Date of Record:** 2026 November 1 **Scope:** Applies to write-behind caches that acknowledge on cache admission and persist at a later flush, where reads can fall through to the durable store. Excludes write-through caches (acknowledgment already waits on persistence), caches with no durable fallback, and systems where every read is pinned to a session that reads its own writes. ## Core Distinction Acknowledgment is not durability. In a write-behind cache the two are separated by the flush interval plus the flush lag; the cache TTL is the promise that the gap stays short enough to mask. **Predicate.** The write-behind acknowledgment is safe if and only if `cache_ttl >= flush_interval + max_flush_lag`, measured on the same write path that admits the write. **Failure.** When the predicate fails, an acknowledged write can expire from the cache before the durable store has received it. A subsequent read falls through to the durable store, which still holds pre-write state. The write was acknowledged and then reverted, with no error at any layer. **Asymmetry.** A stale read returns old data and is surfaced by comparison against a source of truth. A reverted write erases new data and is indistinguishable from the write never having occurred. The second is the failure mode; the first is the one dashboards monitor for. ## What Was Observed **Signature.** Write admission logged at millisecond latency. Read of the same key after TTL expiry returns the pre-write value. No error, no failed batch, no retry storm. The client reports that its write disappeared. **Trace.** TTL set to 10s, flush interval 30s, p99 flush lag 8s. Write admitted at t=0. Cache entry expires at t=10. Read at t=12 misses the cache and falls through to the durable store. Durable store has not yet flushed (next flush at t=30). The read returns pre-write state. If read-triggered re-fill is enabled, the cache is repopulated with the pre-write value at t=12, overwriting the write that was still present at t=10 — the write is not merely invisible, it is actively overwritten. **Why monitoring stays green.** Write success rate is 100 percent: admission succeeded. Read latency improves: TTL expiry turns durable reads into cache hits. The failure is not in any counter the pipeline reports, because every emitted row is individually valid and only the population is wrong. ## Condition of Validity Acknowledged writes remain durable only while at least one of the following holds: 1. **Numeric.** `cache_ttl >= flush_interval + max_flush_lag`, measured on the same write path — not a fleet-wide average. 2. **Pinned.** Every read is pinned to a session that reads its own writes (session affinity, write marker, or read-through-with-write-version). 3. **Topology.** The cache is write-through: acknowledgment waits on durable persistence, and TTL is irrelevant to durability. Mixing 1 and 2 — write-behind with session affinity for some clients but not others — produces the worst case: correct for the pinned cohort, silently reverted for the rest, and indistinguishable from each other in aggregate metrics. ## Decision Boundary - `cache_ttl >= flush_interval + max_flush_lag`: acknowledged writes durable. Monitor the flush lag tail; TTL must track it. - `cache_ttl < flush_interval + max_flush_lag` with write-behind: acknowledged writes at risk. Either raise TTL, paying staleness for readers, or switch to write-through for the affected keys. - TTL chosen from a fleet-wide average while the write path is tail-heavy: treat as the previous case regardless of the numbers. The average does not describe the tail. - Read-triggered re-fill enabled with write-behind: even a correct TTL leaves the overwrite race. Disable re-fill on keys with in-flight writes, or serialize write and fill per key. ## Operational Effect The invariant is a comparison between two constants maintained by different teams, neither of which typically knows about the other, and neither of which is revisited when the write path slows down. Remediation is a coupling, not a patch: `cache_ttl` must be derived from `flush_interval + max_flush_lag` measured on the same path, and re-derived when the flush path changes. **Trap.** The naive fix is to raise TTL. That extends the window but does not close it: the predicate is a comparison against a moving `max_flush_lag`, not against a fixed number. A configuration safe at TTL 60s becomes unsafe when a batch backfill pushes flush lag to 70s. The correct fix is a guard that verifies TTL against measured flush lag at write admission — not a one-time configuration change. ```python from dataclasses import dataclass class CacheDurabilityError(RuntimeError): """Raised when a write-behind cache TTL cannot cover the flush delay it promises to mask.""" @dataclass(frozen=True) class WriteBehindGate: cache_ttl_s: int flush_interval_s: int max_flush_lag_s: int read_refill_enabled: bool = True def verify(self) -> bool: required = self.flush_interval_s + self.max_flush_lag_s if self.cache_ttl_s < required: raise CacheDurabilityError( f"ttl {self.cache_ttl_s}s < required {required}s " f"(flush {self.flush_interval_s}s + max lag {self.max_flush_lag_s}s)" ) return True def dispatch_write_path(gate: WriteBehindGate) -> str: try: gate.verify() except CacheDurabilityError: return "write-through" if gate.read_refill_enabled: return "write-behind+serialize-fill" return "write-behind" ```