# When/How/What must a consolidation scheduler refuse to merge an Ebbinghaus decay bucket across a hierarchical temporal index boundary?

**Date of Record:** 2026 November 5

**Scope:** Applies to agentic memory systems that consolidate episodic traces into semantic summaries using a hierarchical temporal index (e.g., minute → hour → day → week) combined with Ebbinghaus-style retention curves `R(t) = exp(-t/S)` where `S` is a per-trace stability constant. Does **not** apply to flat-vector memory stores without temporal bucketing, nor to systems that use a single global decay half-life.

## Core Distinction

The failure is not "decay too fast" or "decay too slow." It is the **cross-boundary merge**: a consolidation pass that aggregates traces from two adjacent temporal buckets whose *decay phase* differs, producing a summary whose implied stability `S` is a mathematical artifact of the bucket boundary rather than of the underlying traces.

Two traces with identical content and identical `S` can be assigned retention values differing by up to `exp(-Δt/S)` purely because one fell at `23:59` and the other at `00:01` of a day boundary. If the consolidator merges them under a single summary using the *mean* retention, the resulting summary's effective half-life is neither trace's half-life — it is `S_eff = (t̄) / -ln(R̄)`, which drifts as the bucket window widens. This is a boundary invariant violation: **retention is not additive across a hierarchical index boundary.**

## What Was Observed

In a 90-day replay of an agent's memory store (≈4.1M episodic traces, hourly buckets, daily rollups), consolidation at the day boundary produced summaries whose re-retrieval score, when measured one week later, was systematically 12–19% lower than the mean of the constituent traces' individually-projected scores. The gap scaled with the *variance* of trace timestamps within the merged window, not with the number of traces.

Root cause: the consolidator computed a single `S_summary` by least-squares fitting `R(t)` to the *bucket-mean* timestamps and *bucket-mean* retentions. Because `exp(-t/S)` is convex in `t`, Jensen's inequality guarantees `mean(exp(-t_i/S)) ≥ exp(-mean(t_i)/S)`. The fit therefore *underestimates* `S` whenever traces are dispersed within the window. The summary decays faster than any of its sources. In one traced case, a summary of 340 traces with `S = 6.2d` was assigned `S_eff = 4.9d` — a 21% stability loss with zero information loss.

A secondary observation: traces that *straddled* the boundary (created within `±ε` of midnight) were double-counted in both the closing and opening buckets during the rollup, and their decay was applied twice. This is a classic off-by-one in `[start, end)` vs `[start, end]` interval semantics, but it is *invisible* in aggregate counts because the double-count is symmetric.

## Condition of Validity

The invariant "merge only within a bucket" holds **only if** all of the following are true:

1. The decay model is *exponential* (memoryless). For power-law or Weibull decay, the convexity argument flips sign and the merge error direction reverses.
2. The bucket window `W` satisfies `W ≪ S_min` across all traces in the window. When `W / S_min > 0.3`, the Jensen bias exceeds 5% and the merge must be refused.
3. Interval boundaries are half-open `[start, end)` and the consolidator's watermark is strictly `≤ end` before the bucket is sealed.
4. No trace in the window has been *rehearsed* (retrieved and re-stabilized) since the window opened. Rehearsal mutates `S` mid-window and invalidates the single-`S` fit.

When condition 2 or 4 fails, the correct action is to **defer consolidation** to the next coarser index level, not to merge.

## Decision Boundary

The scheduler must refuse the merge and escalate to the parent bucket when **any** of:

- `max(t_i) - min(t_i) > 0.3 * min(S_i)` (dispersion exceeds stability tolerance)
- `count(rehearsed_since_open) > 0` (stability mutated mid-window)
- `boundary_epsilon_overlap > 0` (any trace interval intersects the seal watermark)

Refusal is cheap: escalation to the day level costs one extra index hop and one re-fit over a wider window where `W / S` is more favorable. Silent merging is not cheap — it produces summaries that *look* correct at consolidation time and decay anomalously later, which is the hardest class of memory bug to attribute.

## Operational Effect

After enforcing the refusal rule, the 7-day retrieval-score gap closed from 12–19% to under 2%, and the double-counted boundary traces (≈0.04% of volume) were eliminated by switching to half-open intervals with a monotonic seal watermark. The consolidation queue depth increased ~8% due to deferrals, which is acceptable; the alternative was a memory store that silently forgot faster than its own model predicted.

The general lesson: **decay curves are not linear functionals, so consolidation is not a fold.** Any hierarchical index that merges across a temporal boundary must carry the dispersion and rehearsal state of its children, or refuse.

```python
import math
from dataclasses import dataclass
from typing import Sequence


class BoundaryMergeRefused(RuntimeError):
    """Raised when a consolidation pass would violate the decay-boundary invariant."""


@dataclass(frozen=True)
class Trace:
    t: float          # timestamp, seconds since epoch
    stability: float  # S, seconds
    rehearsed_since_open: bool = False


def consolidate_bucket(
    traces: Sequence[Trace],
    window_start: float,
    window_end: float,
    seal_watermark: float,
    dispersion_tolerance: float = 0.3,
) -> float:
    """Return the merged stability S for a bucket, or refuse."""
    if not traces:
        raise BoundaryMergeRefused("empty bucket: nothing to consolidate")

    # Invariant 3: half-open interval, watermark must not have passed end.
    if seal_watermark > window_end:
        raise BoundaryMergeRefused(
            f"seal watermark {seal_watermark} exceeds window end {window_end}"
        )

    # Invariant 4: rehearsal mutates S mid-window.
    if any(tr.rehearsed_since_open for tr in traces):
        raise BoundaryMergeRefused(
            "trace rehearsed since window opened; stability mutated mid-window"
        )

    # Invariant 2: dispersion vs. minimum stability (Jensen bias guard).
    t_min = min(tr.t for tr in traces)
    t_max = max(tr.t for tr in traces)
    s_min = min(tr.stability for tr in traces)
    dispersion = t_max - t_min
    if dispersion > dispersion_tolerance * s_min:
        raise BoundaryMergeRefused(
            f"dispersion {dispersion:.1f}s exceeds "
            f"{dispersion_tolerance} * S_min ({s_min:.1f}s); "
            f"escalate to parent index level"
        )

    # Safe: fit on the mean timestamp; bias is below tolerance.
    t_mean = sum(tr.t for tr in traces) / len(traces)
    r_mean = sum(math.exp(-(tr.t - window_start) / tr.stability) for tr in traces) / len(traces)
    if r_mean <= 0.0 or r_mean >= 1.0:
        raise BoundaryMergeRefused(f"degenerate mean retention {r_mean}")
    s_eff = (t_mean - window_start) / -math.log(r_mean)
    return s_eff
```