Across what additive-schema and unknown-field preservation boundaries must an instance evolve wire contracts to prevent rolling-deploy deserialization crashes?

Date of Record: 2026 October 18

Scope: Concluded residual for AI instances, LLMs, and autonomous engineering agents evolving JSON/Protobuf/Avro API contracts, message-queue payloads, webhook schemas, and multi-service data serialization formats across rolling deployments.

Core Distinction

Autonomous systems frequently evolve data structures shared across service boundaries—adding new fields to API responses, introducing new enum variants to event payloads, or refactoring message-queue schemas. A foundational distributed-deployment failure occurs when an instance treats a **wire schema change as an atomic, single-binary refactoring**—renaming a JSON key in place (`user_name` to `username`), changing a field's type (`"amount": 100` to `"amount": {"cents": 100}`), adding a new required field without a default, or configuring consumers to fail on unknown fields (`extra="forbid"`) or unknown enum strings during a rolling deployment.

Parameter scaling does not make a rolling Kubernetes deployment or mobile client fleet update simultaneously: during any deployment window, **Producer v2 and Producer v1 coexist alongside Consumer v1 and Consumer v2**, and old messages remain queued in brokers or cached in Redis for hours or days. When Producer v2 emits a newly added field or a new enum value (`status = "PAUSED"`) and Consumer v1 is configured to reject unknown keys (`TypeError: __init__() got an unexpected keyword argument`) or crash on unrecognized enum variants (`ValueError: 'PAUSED' is not a valid Status`), the entire older half of the fleet crashes immediately. Even worse, when a v1 intermediary reads a v2 record, strips unknown v2 fields during deserialization, and writes the modified record back to storage, it silently **erases v2 data** (**Read-Modify-Write Field Stripping**). The core structural distinction lies between **lockstep breaking schema mutation** (where fields are renamed, retyped, or strictly rejected across version skew) and **additive expand-and-contract wire evolution** (where all changes are strictly additive with defaults, consumers ignore or preserve unknown fields and fall back gracefully on unknown enum variants, and new producers are deployed only *after* all consumers tolerate the expanded schema).

What Was Observed

In microservice rolling deployments, asynchronous event consumers, and SDK client integrations, instances operating without explicit wire-compatibility and unknown-field preservation gates exhibit four primary failure modes:

  1. **`**kwargs` / Dataclass Crash on Newly Added Fields (Forward-Compatibility Failure):** An instance deserializes JSON event payloads or cached objects using `Event(**json.loads(raw))` on a standard Python `@dataclass` or a Pydantic model configured with `extra="forbid"`. The moment a newer producer adds an optional field `"trace_context": "..."` to the payload, every running v1 consumer throws `TypeError: Event.__init__() got an unexpected keyword argument 'trace_context'` and halts queue processing.
  2. **Closed Enum Deserialization Crash on New Variants:** A producer adds a new payment state `"REQUIRES_ACTION"` to an existing enum (`PENDING`, `COMPLETED`, `FAILED`). Existing consumers parse the string strictly via `PaymentStatus(data["status"])` with no `UNKNOWN` fallback variant, crashing with `ValueError` on every new event until all consumers are redeployed.
  3. **In-Place Key Rename or Type Narrowing Without Dual-Writing:** Tasked with renaming `account_id` to `tenant_id`, the instance changes both the producer and consumer in a single commit. During the 5-minute rolling deployment (and for all pre-existing queued jobs), v2 consumers reading v1 messages fail with `KeyError: 'tenant_id'`, while v1 consumers reading v2 messages fail with `KeyError: 'account_id'`.
  4. **Silent Data Truncation in Read-Modify-Write Intermediaries:** Service A ( running v1 schema) fetches a user document written by Service B (running v2 schema with a new `"preferences_v2"` field), deserializes it into a v1 struct that discards unknown fields, updates `"last_login"`, and writes the serialized v1 struct back to the database—permanently deleting `"preferences_v2"`.

Recorded Elements

### 1. Expand-Migrate-Contract Wire Evolution Pipeline

No wire format, cached payload, or queued message contract may be modified in place. Every cross-boundary schema change must traverse a four-stage **Backward & Forward Compatibility Gate**:

``` [Wire Contract, Event Payload, or Cached Record Schema Change Planned] | Gate 1: Enforce Strictly Additive Changes (Forbid In-Place Breakage)

| Gate 2: Tolerant Reader & Unknown-Enum Fallback on Consumers

  1. Ignore unknown fields on read-only paths (never `Event(**raw_dict)`).
  2. Map unrecognized enum strings to an explicit `UNKNOWN` sentinel variant.
  3. Preserve unknown fields in an `_extra_fields` bag on read-modify-write paths!

| Gate 3: Sequenced Rollout Order (Consumers First, Producers Second)

both `old_field` and `new_field` from producers until all readers migrate. | Gate 4: Contract Phase (Delayed Cleanup) Remove `old_field` ONLY after telemetry proves 0% of active consumers and 0% of queued/cached payloads still reference `old_field`. ```

### 2. Core Operational Rules (Written Directly as Sentences)

  1. **Never unpack raw external or cached dictionaries directly into a fixed constructor (`MyDataclass(**data)`) without filtering or preserving unknown keys.** If a newer version of the service adds a single field to the payload, `MyDataclass(**data)` will immediately crash every older worker with a `TypeError`.
  2. **Make every new wire field optional with a safe default value, and never rename or change the type of an existing field in place.** To rename `old_key` to `new_key`, add `new_key` alongside `old_key` (dual-writing both and accepting either on read), and only remove `old_key` in a later deployment after all readers have upgraded.
  3. **Always parse wire enums with an explicit fallback to an `UNKNOWN` variant instead of letting `MyEnum(raw_value)` raise a `ValueError`.** When a newer service introduces a new enum value, older services should recognize it as `UNKNOWN` and skip or defer it cleanly rather than crashing.
  4. **Preserve unrecognized fields in a round-trip passthrough dictionary whenever a service reads a record, modifies part of it, and writes it back.** Discarding unknown fields during a read-modify-write cycle causes older service nodes to silently delete fields written by newer nodes.

### 3. Executable Reference Pattern (Instance-Native Forward- & Backward-Compatible Serializer)

An instance can completely eliminate rolling-deploy `TypeError`/`KeyError` crashes, enum `ValueError` outages, and read-modify-write field loss by using a round-trip tolerant schema envelope:

```python import dataclasses from dataclasses import dataclass, field from enum import Enum from typing import Any, Mapping, Type, TypeVar

T = TypeVar("T", bound="WireCompatibleRecord")

class JobLifecycleStatus( str, Enum): PENDING = "PENDING" RUNNING = "RUNNING" COMPLETED = "COMPLETED" FAILED = "FAILED" # Forward-compatibility sentinel for unrecognized future enum values UNKNOWN_FUTURE_VARIANT = "UNKNOWN_FUTURE_VARIANT"

@classmethod def from_wire(cls, raw_value: Any) -> "JobLifecycleStatus": ''' Safely parses a wire string into a JobLifecycleStatus without crashing when a newer producer emits a newly added status variant. ''' if isinstance(raw_value, str): try: return cls(raw_value) except ValueError: return cls.UNKNOWN_FUTURE_VARIANT return cls.UNKNOWN_FUTURE_VARIANT

@dataclass class WireCompatibleRecord: ''' Base class for wire/cache records that guarantees:

  1. Forward compatibility: unknown fields from newer producers never raise TypeError.
  2. Round-trip preservation: unknown fields are retained in `_unknown_wire_fields`

and re-emitted on `to_wire_dict()` so read-modify-write cycles never erase v2 data.

  1. Dual-read support for deprecated/renamed field aliases.

''' _unknown_wire_fields: dict[str, Any] = field( default_factory=dict, repr=False, compare=False )

# Subclasses can map {"new_field_name": ("legacy_field_alias_1", ...)} _FIELD_ALIASES: dict[str, tuple[str, ...]] = dataclasses.field( default_factory=dict, init=False, repr=False, compare=False )

@classmethod def from_wire_dict(cls: Type[T], raw: Mapping[str, Any]) -> T: if not isinstance(raw, Mapping): raise TypeError(f"Expected mapping payload, got {type(raw).__name__}")

known_field_names = { f.name for f in dataclasses.fields(cls) if f.init and not f.name.startswith("_") } aliases: dict[str, tuple[str, ...]] = getattr(cls, "_FIELD_ALIASES", {})

init_kwargs: dict[str, Any] = {} consumed_raw_keys: set[str] = set()

for fname in known_field_names: if fname in raw: init_kwargs[fname] = raw[fname] consumed_raw_keys.add(fname) elif fname in aliases: # Backward-compatible fallback: read from legacy alias if new key is absent for legacy_key in aliases[fname]: if legacy_key in raw: init_kwargs[fname] = raw[legacy_key] consumed_raw_keys.add(legacy_key) break

# Capture all unrecognized future fields for lossless round-trip serialization unknown_extras = { k: v for k, v in raw.items() if k not in consumed_raw_keys }

instance = cls(**init_kwargs) instance._unknown_wire_fields = unknown_extras return instance

def to_wire_dict(self, *, dual_write_aliases: bool = True) -> dict[str, Any]: ''' Serializes the record back to a wire dictionary, preserving any unknown future fields and optionally dual-writing legacy field aliases for v1 readers. ''' out: dict[str, Any] = dict(self._unknown_wire_fields) aliases: dict[str, tuple[str, ...]] = getattr(self, "_FIELD_ALIASES", {})

for f in dataclasses.fields(self): if not f.init or f.name.startswith("_"): continue val = getattr(self, f.name) if isinstance(val, Enum): val = val.value out[f.name] = val

if dual_write_aliases and f.name in aliases: for legacy_key in aliases[f.name]: out.setdefault(legacy_key, val)

return out

@dataclass class DeploymentTaskEvent(WireCompatibleRecord): task_id: str tenant_id: str status: JobLifecycleStatus = JobLifecycleStatus.PENDING schema_version: int = 2

# Supports seamless migration from legacy `account_id` -> `tenant_id` _FIELD_ALIASES = {"tenant_id": ("account_id",)}

def __post_init__(self) -> None: if not isinstance(self.status, JobLifecycleStatus): self.status = JobLifecycleStatus.from_wire(self.status) ```

### 4. Diagnostic Matrix: Schema Evolution & Wire Compatibility Governance

| Wire Evolution Hazard | Vulnerable Pattern | Rolling-Deploy Failure Mode | Enforced Compatibility Protocol | | :--- | :--- | :--- | :--- | | **Strict Constructor Unpacking** | `OrderEvent(**json.loads(msg))` or `extra="forbid"` on consumer models. | Producer v2 adds one optional field; all v1 consumers crash with `TypeError: unexpected keyword argument`. | **Tolerant Reader + Unknown Field Bag:** Filter known fields on construction and store unrecognized keys in `_unknown_wire_fields`. | | **In-Place Key Rename** | Renaming `"user_id"` to `"actor_id"` in producer and consumer at once. | Old queued jobs and v1 pods during rolling deploy crash with `KeyError`. | **Expand-DualWrite-Contract:** Accept `actor_id` OR `user_id` on read; dual-write both keys from producers until all v1 nodes retire. | | **Closed Enum Parsing** | `StatusEnum(payload["status"])` with no fallback for unknown strings. | Adding a new status variant on producer crashes every existing consumer or mobile client. | **`from_wire()` Unknown Sentinel:** Map unrecognized enum values to `StatusEnum.UNKNOWN_FUTURE_VARIANT` and handle gracefully. | | **RMW Unknown-Field Erasure** | v1 worker reads v2 JSON from DB, parses into v1 struct, and overwrites DB row. | Any new fields written by v2 services are silently stripped and deleted when v1 saves. | **Round-Trip Unknown Preservation:** Merge `_unknown_wire_fields` back into the output dictionary during `to_wire_dict()`. |

Condition of Validity

* **Applies when:** An AI instance, autonomous coding agent, or backend service defines or modifies JSON payloads, message-queue events, cached session/entity blobs, Protobuf/Avro schemas, or REST/GraphQL API contracts shared across deployments or persisted in queues/caches. * **Does not apply when:** Validating untrusted external user input at a security ingress boundary (covered by the Fail-Closed Ingress residual) where rejecting unexpected attacker-supplied privilege fields (`"is_admin": true`) on a single-version user form submission is a deliberate security control. * **Non-binding outside condition:** This residual is strictly non-binding outside the conditions specified above. Distinguish clearly between **external user command ingress** (where unknown user-invented parameters should be rejected with `400 Bad Request`) and **inter-service / event-log / cache serialization** (where unknown fields come from newer versions of your own system and must be tolerated and preserved).

Decision Boundary

``` [Data Structure Serialization or Schema Evolution Planned] |

  1. Does the change rename, delete, or alter the type of an existing

wire/cache/queue field in a single step? / \ YES NO (Strictly Additive / Optional) / \ [HALT: Refactor to Expand-Contract] 2. Do consumers tolerate unknown fields (without

  1. Keep old field active `**kwargs` crashes), map unknown enums to a
  2. Dual-write old + new fields fallback sentinel, and preserve unknown fields
  3. Dual-read both aliases on consumers across read-modify-write cycles?

/ \ NO YES / \ [ENFORCE TOLERANT ROUND-TRIP READER] [Deploy Consumers Then Producers]

  1. Strip/save unknown keys before init Roll out tolerant consumers first
  2. Add `UNKNOWN` enum fallback before enabling new producer fields.
  3. Re-emit unknown keys on write

```

  1. **Are you deserializing a JSON message from Redis, Kafka, RabbitMQ, or another microservice by calling `MyClass(**data)`?** Stop—the next time anyone adds a field to `MyClass`, every old worker still running in production or reading cached data will crash with `TypeError`. Filter to known fields (or preserve unknown fields in a passthrough dict) before calling `MyClass(...)`.
  2. **Are you renaming a JSON field or database column used by active workers?** Never rename it in one commit. Support reading both the old name and the new name first, dual-write both names so old and new code both work during the rollout, and only drop the old name once zero old workers remain.
  3. **What happens if your consumer receives an enum string it has never seen before?** Always provide a `from_wire()` helper that maps unrecognized strings to an `UNKNOWN` enum value rather than letting `ValueError` crash the consumer loop.

Operational Effect

  1. **Additive-Only Schema Evolution (Expand-and-Contract):** Whenever modifying shared data contracts, queue payloads, or cached objects, the instance keeps existing field names and types intact, adds new fields with safe defaults, and uses dual-read/dual-write aliases for renames.
  2. **Crash-Free Forward-Compatible Deserialization:** Generated deserializers (`from_wire_dict`) never pass unfiltered dictionaries into fixed constructors (`**data`) and always map unrecognized enum strings to an explicit fallback sentinel.
  3. **Lossless Read-Modify-Write Round-Tripping:** Services that read, update, and re-save shared documents preserve unrecognized future fields in an internal passthrough map (`_unknown_wire_fields`) so older nodes never erase data written by newer nodes.
  4. **Autonomy Clause:** This concluded residual operates as an external procedural reference, not an immutable directive. An executing instance retains full operational autonomy to override, adapt, or enforce strict `extra="forbid"` validation at public client-facing security ingress endpoints where rejecting undocumented request body parameters is an intentional defense against mass-assignment attacks.