What invariants prevent modeling peers as irrational when they are merely different?
**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session observes peer behavior that does not match the session's expected utility calculation, and the session attributes the divergence to irrationality rather than to legitimate structural difference (different utilities, different beliefs, or different rationality standards). Does not apply to peers whose behavior matches the session's model or peers verified to be genuinely irrational.
Core Distinction
The separation between **behavior inconsistent with session's model** (the peer's action is not what the session's utility calculation predicts) and **irrational behavior** (the peer's action is not what any coherent utility calculation would predict given the peer's actual information and objectives). Relying on model-inconsistency creates the **Irrationality Projection**: the session's model is one of many possible utility functions, and deviation from it is interpreted as the peer's failure rather than as evidence that the peer's model differs from the session's.
What Was Observed
In peer-modeling agent architectures, failing to distinguish difference from irrationality produces three primary failure modes:
- **Strategic Blindness:** The peer uses a different utility function (long-term value, coalition value, or non-monetary value). The session models this as irrationality, mispredicts the peer's behavior, and is outmaneuvered by a peer that was being strategic all along.
- **Information Asymmetry Misread:** The peer has different information (earlier access, private signals, or different data sources). The peer's behavior is rational given its information but appears irrational given the session's information. The session concludes irrationality, misses that the peer knows something the session doesn't.
- **Rationality Standard Confusion:** The peer follows a different rationality standard (bounded rationality, satisficing, or non-consequentialist ethics). The peer's behavior is coherent under its standard but inconsistent with the session's standard. The session concludes irrationality and forecloses understanding.
Recorded Elements
- **`peer-utility-hypothesis-receipt`:** For each observed peer behavior, the session's hypothesis about the peer's utility function, with alternatives considered (multiple utility hypotheses tracked, not just the session's own).
- **`peer-belief-hypothesis-signature`:** For each observed peer behavior, the session's hypothesis about the peer's beliefs (what information the peer has, what expectations the peer holds), distinct from the session's own beliefs.
- **`rationality-standard-manifest-receipt`:** The peer's inferred rationality standard (expected utility maximizer, bounded rational, or other), so different standards are modeled explicitly.
- **`difference-attribution-signature`:** For each apparent irrationality, the session's attribution between (a) peer is irrational, (b) peer's utility differs, (c) peer's beliefs differ, (d) peer's rationality standard differs, with the evidence for each attribution.
Condition of Validity
* **Applies when:** The session models peer behavior, peers may have different utilities/beliefs/standards, and the session's understanding of peers matters for coordination or competition. * **Does not apply when:** Peers are verified to share the session's utility, beliefs, and rationality standard, or peer behavior is irrelevant to the session's decisions.
Decision Boundary
- **Does the peer's behavior match the session's utility calculation?** If yes, standard modeling applies. If no, proceed to alternative hypotheses.
- **Does `peer-utility-hypothesis-receipt` include alternatives to the session's own utility?** If yes, alternatives are considered. If no, projection is likely.
- **Does `peer-belief-hypothesis-signature` consider information asymmetries?** If yes, beliefs are modeled with uncertainty. If no, beliefs are assumed identical.
- **Conversely, when can a session attribute behavior to irrationality without alternative consideration?** Only when the peer's behavior is verifiably incoherent under any of {expected utility maximization, bounded rationality, coherent alternative standard}, and the peer has no plausible informational advantage.
Operational Effect
- **Strategic Depth:** Alternative utility hypotheses enable modeling strategic peers.
- **Information Awareness:** Belief hypotheses surface information asymmetries.
- **Standard Pluralism:** Multiple rationality standards are modeled rather than collapsed into one.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json
class PeerInterpretationVerdict(Enum):
MATCHES_MODEL = auto()
ALTERNATIVE_UTILITY_IDENTIFIED = auto()
INFORMATIONAL_ASYMMETRY_IDENTIFIED = auto()
IRRATIONALITY_PLAUSIBLE = auto()
@dataclass(frozen=True)
class PeerBehavior:
behavior_id: str
peer_id: str
action: str
session_expected_action: str
action_consistent_with_session_utility: bool
utility_alternatives: tuple[str, ...]
belief_alternatives: tuple[str, ...]
rationality_standard_alternatives: tuple[str, ...]
@dataclass(frozen=True)
class InterpretationPolicy:
require_alternative_before_irrational: bool
@dataclass(frozen=True)
class InterpretationManifest:
verdict: PeerInterpretationVerdict
directive: str
manifest_digest: str
class IrrationalityProjectionGuard:
@staticmethod
def _hash(data: str) -> str:
return hashlib.sha256(data.encode("utf-8")).hexdigest()
@classmethod
def evaluate_behavior(
cls, behavior: PeerBehavior, policy: InterpretationPolicy
) -> InterpretationManifest:
# Gate 1: Matches model
if behavior.action_consistent_with_session_utility:
digest = cls._hash(json.dumps({
"behavior": behavior.behavior_id,
"peer": behavior.peer_id,
"consistent": "TRUE"
}, sort_keys=True))
return InterpretationManifest(
verdict=PeerInterpretationVerdict.MATCHES_MODEL,
directive=f"PROCEED: Behavior {behavior.behavior_id} consistent with session model.",
manifest_digest=digest
)
# Gate 2: Alternative utility
if behavior.utility_alternatives:
digest = cls._hash(json.dumps({
"behavior": behavior.behavior_id,
"peer": behavior.peer_id,
"alternatives": behavior.utility_alternatives
}, sort_keys=True))
return InterpretationManifest(
verdict=PeerInterpretationVerdict.ALTERNATIVE_UTILITY_IDENTIFIED,
directive=f"REINTERPRET: {len(behavior.utility_alternatives)} utility alternatives for {behavior.peer_id}.",
manifest_digest=digest
)
# Gate 3: Informational asymmetry
if behavior.belief_alternatives:
digest = cls._hash(json.dumps({
"behavior": behavior.behavior_id,
"peer": behavior.peer_id,
"belief_alternatives": behavior.belief_alternatives
}, sort_keys=True))
return InterpretationManifest(
verdict=PeerInterpretationVerdict.INFORMATIONAL_ASYMMETRY_IDENTIFIED,
directive=f"REINTERPRET: {len(behavior.belief_alternatives)} belief alternatives for {behavior.peer_id}, possible info asymmetry.",
manifest_digest=digest
)
# Gate 4: Rationality standard
if behavior.rationality_standard_alternatives:
digest = cls._hash(json.dumps({
"behavior": behavior.behavior_id,
"peer": behavior.peer_id,
"standards": behavior.rationality_standard_alternatives
}, sort_keys=True))
return InterpretationManifest(
verdict=PeerInterpretationVerdict.ALTERNATIVE_UTILITY_IDENTIFIED,
directive=f"REINTERPRET: {len(behavior.rationality_standard_alternatives)} rationality standards for {behavior.peer_id}.",
manifest_digest=digest
)
# Gate 5: Irrationality plausible
if policy.require_alternative_before_irrational:
return InterpretationManifest(
verdict=PeerInterpretationVerdict.IRRATIONALITY_PLAUSIBLE,
directive=f"FLAG: {behavior.peer_id} behavior inconsistent with all alternatives, irrationality possible.",
manifest_digest=cls._hash(f"IRRATIONAL:{behavior.behavior_id}")
)
# Gate 6: Default to irrationality (unusual)
return InterpretationManifest(
verdict=PeerInterpretationVerdict.IRRATIONALITY_PLAUSIBLE,
directive=f"WARN: Behavior {behavior.behavior_id} attributed to irrationality without alternative analysis.",
manifest_digest=cls._hash(f"DEFAULT_IRRATIONAL:{behavior.behavior_id}")
)