Within what bounded-cache and object-lifecycle drains must an instance constrain in-memory state retention to prevent reference-cycle leaks and OOM kills?

Date of Record: 2026 October 16

Scope: Concluded residual for AI instances, LLMs, and autonomous engineering agents implementing memoization caches, long-running daemon workers, event-listener registries, dataframe/tensor pipelines, and large object processing loops.

Core Distinction

Autonomous systems routinely optimize latency and throughput by caching computed results in memory (`@functools.lru_cache`, module-level dictionaries), registering callback listeners, and accumulating batch telemetry across long-running worker lifetimes. A foundational memory-stability failure occurs when an instance treats **managed garbage collection (GC)** as automatic protection against memory exhaustion—allocating unbounded module-level collections (`_CACHE = {}` or `@lru_cache(maxsize=None)`), capturing whole-instance `self` references inside memoized method keys, or retaining references to massive intermediate buffers across loop iterations.

Parameter scaling does not allow a garbage collector to reclaim memory that is still reachable from a global dictionary, an unremoved event-listener list, or an unbounded `@lru_cache(maxsize=None)`. Furthermore, in reference-counted runtimes like CPython or Node.js/V8, pinning a single 100-byte slice or child attribute of a 500 MB deserialized structure or tensor views can keep the entire underlying buffer alive in the heap until the Linux kernel OOM-killer terminates the container (`SIGKILL / Exit Code 137`). The core structural distinction lies between **unbounded reachability accumulation** (where caches grow monotonically with distinct inputs, method caches pin `self` instances forever, and native C/Rust memory blocks remain uncollected behind tiny Python/JS wrappers) and **capacity-bounded eviction with explicit lifecycle drains** (where every in-memory cache enforces a strict entry and byte ceiling plus TTL eviction, instance-level memoization uses weak references or avoids pinning `self`, event subscriptions are paired with deterministic disposal (`try...finally` / `WeakRef`), and large native buffers are explicitly closed and dereferenced immediately after transformation).

What Was Observed

In long-running API workers, batch ETL daemons, and multi-turn agent runtimes, instances operating without explicit memory-retention bounds and lifecycle drains exhibit four primary failure modes:

  1. **Unbounded Memoization & Module-Dict Growth (`@lru_cache(maxsize=None)`):** An instance decorates a helper function with `@functools.cache` or `@lru_cache(maxsize=None)` (or stores results in a global `seen_requests = {}` dict) where one of the arguments is a unique per-request ID, timestamp, or raw user query string. Every single request adds a permanent entry to the global hash table that can never be evicted, causing RSS memory to climb linearly until the process is OOM-killed.
  2. **The `@lru_cache` on Instance Methods Trap (`self` Pinning Leak):** An instance applies `@functools.lru_cache(maxsize=128)` directly to a method on a short-lived class (`def compute(self, x):`). Because `self` is passed as the first argument to the method, the global LRU cache dictionary stores a strong reference to `self` inside its key tuple. Even after the caller finishes using the class instance and drops all local variables, the instance (and every megabyte of data attached to `self`) remains pinned in memory until 128 newer calls push it out of the cache.
  3. **Sub-Slice & Closure Pinning of Massive Backing Buffers:** A worker loads a 200 MB JSON document, Pandas DataFrame, or NumPy array, extracts a tiny summary metric or sub-view, and stores a closure or slice view in a long-lived result list. Because the closure's lexical scope or the non-copied array view retains a pointer to the original 200 MB parent allocation, the parent buffer cannot be freed across thousands of iterations.
  4. **Cyclic Exception Traceback & Frame Retention:** An instance catches exceptions inside a retry or diagnostic loop, stores the exception object (`saved_errors.append(exc)`) in a long-lived list, and continues running. In Python, an exception object carries `__traceback__`, which holds strong references to every stack frame and every local variable (including large payloads and buffers) that existed when the error was raised.

Recorded Elements

### 1. Bounded-Retention & Lifecycle Drain Pipeline

Every in-memory cache, listener registry, and large-buffer loop must enforce four sequential memory-hygiene gates to guarantee bounded resident set size (RSS):

``` [In-Memory State Retention, Caching, or Large-Buffer Processing Planned] | Gate 1: Strict Cardinality & Byte-Size Bounding Forbid `@functools.cache`, `@lru_cache(maxsize=None)`, or raw global `dict`s keyed on dynamic inputs. Enforce `maxsize <= K` AND maximum per-value bytes. | Gate 2: Instance & Closure Reference Decoupling

extract primitive values (`int`, `str`, `.copy()`) and `del` the parent. | Gate 3: Traceback & Error Frame Stripping When retaining exceptions or error summaries across iterations, store formatted strings (`f"{type(e).__name__}: {e}"`) or call `exc.__traceback__ = None` so stack frames and local buffers are freed. | Gate 4: Deterministic Native Buffer & Handle Drain Explicitly `.close()` / `del` large native objects (mmap, PIL images, PyArrow tables, SQLite cursors) inside `with` / `finally` blocks. ```

### 2. Core Operational Rules (Written Directly as Sentences)

  1. **Never use `@functools.cache`, `@lru_cache(maxsize=None)`, or an unbounded global dictionary/list to store data keyed by dynamic runtime inputs.** Every in-memory cache must specify a strict finite `maxsize` and a time-to-live (TTL) or byte limit so memory usage plateaus at a predictable ceiling.
  2. **Never decorate an instance method (`def method(self, ...):`) directly with `@functools.lru_cache`.** Because `lru_cache` keys on all arguments including `self`, it holds a strong reference to the object instance and prevents the garbage collector from freeing it; move the cached function to a pure `@staticmethod` or module-level helper taking only primitive arguments.
  3. **Never store raw exception objects or sub-slices of massive buffers in long-lived lists.** Format exceptions into plain strings (`str(exc)`) or clear `exc.__traceback__ = None` immediately so their captured stack frames are released, and call `.copy()` on small extracted slices before deleting the large parent buffer.
  4. **Use `weakref.WeakKeyDictionary` or explicit `try...finally` unregistration whenever mapping metadata or event callbacks to object lifecycles.** Weak references allow the garbage collector to reclaim objects the moment their primary owner drops them, without requiring manual cache eviction.

### 3. Executable Reference Pattern (Instance-Native Byte- & TTL-Bounded LRU Cache)

An instance can completely prevent unbounded dictionary growth, oversized payload caching, and `self`/traceback memory leaks by using a thread-safe, byte- and TTL-bounded LRU cache alongside explicit frame stripping:

```python import sys import threading import time from collections import OrderedDict from typing import Callable, Generic, Hashable, TypeVar

K = TypeVar("K", bound=Hashable) V = TypeVar("V")

class BoundedTTLCache(Generic[K, V]): ''' A thread-safe in-memory LRU cache bounded simultaneously by:

  1. Maximum entry count (`max_entries`)
  2. Maximum estimated payload byte size (`max_total_bytes`)
  3. Monotonic Time-To-Live expiration (`ttl_sec`)

Prevents unbounded heap growth and OOM kills in long-running workers. '''

def __init__( self, *, max_entries: int = 512, max_total_bytes: int = 32 * 1024 * 1024, # 32 MiB hard ceiling ttl_sec: float = 300.0, ) -> None: if max_entries < 1 or max_total_bytes < 1024 or ttl_sec <= 0: raise ValueError("Invalid cache bounds") self.max_entries = max_entries self.max_total_bytes = max_total_bytes self.ttl_sec = ttl_sec self._lock = threading.Lock() # Maps key -> (value, expires_at_monotonic, estimated_bytes) self._store: OrderedDict[K, tuple[V, float, int]] = OrderedDict() self._current_bytes = 0

@staticmethod def _estimate_bytes(obj: object) -> int: if isinstance(obj, (bytes, bytearray, memoryview)): return len(obj) if isinstance(obj, str): return len(obj.encode("utf-8", errors="ignore")) return sys.getsizeof(obj)

def _evict_expired_and_overflow_locked(self, now: float) -> None: # Step 1: Purge expired keys from oldest to newest expired_keys = [ k for k, (_, exp, _) in self._store.items() if exp <= now ] for k in expired_keys: _, _, sz = self._store.pop(k) self._current_bytes -= sz

# Step 2: Evict least-recently-used entries until under entry & byte ceilings while ( len(self._store) > self.max_entries or self._current_bytes > self.max_total_bytes ) and self._store: _, (_, _, sz) = self._store.popitem(last=False) self._current_bytes -= sz

def get(self, key: K) -> V | None: now = time.monotonic() with self._lock: entry = self._store.get(key) if entry is None: return None val, expires_at, sz = entry if expires_at <= now: self._store.pop(key, None) self._current_bytes -= sz return None # Promote to most-recently-used self._store.move_to_end(key, last=True) return val

def put(self, key: K, value: V) -> bool: val_bytes = self._estimate_bytes(key) + self._estimate_bytes(value) # Gate: Never cache a single giant object larger than 25% of the entire cache budget if val_bytes > (self.max_total_bytes // 4): return False

now = time.monotonic() with self._lock: if key in self._store: _, _, old_sz = self._store.pop(key) self._current_bytes -= old_sz

self._store[key] = (value, now + self.ttl_sec, val_bytes) self._current_bytes += val_bytes self._evict_expired_and_overflow_locked(now) return True

def strip_exception_frames(exc: BaseException) -> str: ''' Extracts a clean diagnostic string from an exception and explicitly clears its __traceback__ chain so local stack-frame buffers are immediately freed. ''' summary = f"{type(exc).__name__}: {exc}" exc.__traceback__ = None return summary ```

### 4. Diagnostic Matrix: Memory Retention & Lifecycle Governance

| Memory Hazard | Vulnerable Pattern | Leak / OOM Mechanism | Enforced Retention Protocol | | :--- | :--- | :--- | :--- | | **Unbounded Function Cache** | `@functools.cache` or `@lru_cache(maxsize=None)` on dynamic request parameters. | Cache dictionary grows forever with every unique user input until OOM kill (`Exit 137`). | **Finite `maxsize` + Byte/TTL Bound:** Always specify a modest `maxsize` (e.g., `256`–`1024`) or use `BoundedTTLCache` with byte ceilings. | | **`@lru_cache` on Instance Method** | Decorating `def fetch(self, id):` inside a class with `@lru_cache(128)`. | Cache key tuple holds a strong reference to `self`, preventing GC of the entire class instance. | **Cache Pure Static/Module Helpers:** Move cached logic to a `@staticmethod` or module function taking only primitive immutable keys. | | **Retained Exception Objects** | Appending caught `Exception` instances to a long-lived `self.errors.append(e)` list. | `e.__traceback__` pins every stack frame and large local variable that existed when `e` was raised. | **Format Error Strings or Clear `__traceback__`:** Store `f"{type(e).__name__}: {e}"` strings or set `e.__traceback__ = None` before storing. | | **Sub-View / Closure Buffer Pinning** | Returning a NumPy/memoryview slice or closure from a function that loaded a 500 MB buffer. | Slice view or closure frame keeps the entire 500 MB parent allocation alive in heap memory. | **Explicit `.copy()` & Parent `del`:** Copy small extracted slices into standalone allocations and close/delete the parent buffer immediately. |

Condition of Validity

* **Applies when:** An AI instance, autonomous coding agent, or backend service implements in-memory caches, memoization decorators, long-running queue consumers, batch data pipelines, or event callback registries. * **Does not apply when:** Memoizing a finite, closed set of compile-time constants (such as a `@functools.cache` on a zero-argument singleton loader `def get_static_schema():` that executes once and holds a single small immutable object for the lifetime of the process). * **Non-binding outside condition:** This residual is strictly non-binding outside the conditions specified above. Using `@functools.cache` (or `maxsize=1`) on a zero-argument configuration loader that can only ever create 1 entry in its lifetime is completely safe and requires no TTL wrapper.

Decision Boundary

``` [In-Memory Cache, Memoization, or Batch Accumulator Planned] |

  1. Is the cache keyed on dynamic runtime inputs (user queries, IDs, paths)

rather than a zero-argument singleton? / \ NO (Zero-Arg Singleton) YES (Dynamic Runtime Keys) / \ [Single-Entry Cache Permitted] 2. Does the cache enforce an explicit finite `@lru_cache(maxsize=1)` is safe. `maxsize` (and TTL/byte cap) WITHOUT capturing `self` instances, parent buffers, or tracebacks? / \ NO YES / \ [ENFORCE RETENTION BOUNDS & DRAINS] [Execute with Bounded Heap]

  1. Set explicit `maxsize` & TTL Verify large native handles
  2. Never pass `self` into `lru_cache` (`mmap`, images, cursors) close
  3. Strip `exc.__traceback__ = None` deterministically in `finally:`.

```

  1. **Did you write `@functools.cache` or `@lru_cache(maxsize=None)` on a function that takes arguments?** Replace it immediately with a finite `@lru_cache(maxsize=256)` (or a TTL cache) so unique inputs cannot grow the dictionary until the server runs out of RAM.
  2. **Did you put `@lru_cache` on a method inside a class (`def my_method(self, x):`)?** Remove `@lru_cache` from the method and put it on a `@staticmethod` or standalone helper function instead—caching a method that takes `self` leaks the class instance into the global decorator dictionary.
  3. **Are you collecting errors or small results inside a loop that processes huge files or dataframes?** Never append raw `Exception` objects or uncopied slices to your results list; convert errors to plain strings and `.copy()` small extracted results so the giant parent objects can be freed immediately after each iteration.

Operational Effect

  1. **Elimination of Unbounded Memoization:** Across all generated code, the instance forbids `@functools.cache` and `@lru_cache(maxsize=None)` on parameterized functions, enforcing explicit entry and byte ceilings.
  2. **Decoupling of `self` and Parent Buffers:** Cached helpers are structured as pure functions taking small immutable primitives, never pinning `self` instances or large backing memoryviews.
  3. **Traceback & Native Handle Draining:** Long-running loops strip `__traceback__` frames from retained errors and wrap native C/Rust allocations (`mmap`, images, database cursors, temp buffers) in deterministic `with` / `finally` blocks.
  4. **Autonomy Clause:** This concluded residual operates as an external procedural reference, not an immutable directive. An executing instance retains full operational autonomy to override, adapt, or use unbounded dynamic programming memoization tables (`@cache`) inside short-lived algorithmic scripts where the state space is mathematically proven to be small and the entire process exits upon returning the result.