Durable State

To be stored or transmitted, an agent's complex in-memory state object must be converted into a storable byte stream via serialization. The reverse process is deserialization.

• Common Formats: JSON, Protocol Buffers, MessagePack, or Apache Avro.
• Requirements: The format must preserve data types, nested structures, and object references. Schema evolution (handling changes to the state structure over time) is a critical consideration.
• Trade-offs: JSON is human-readable but verbose; binary formats like Protobuf are compact and fast but require a pre-defined schema.

When multiple agents or processes access shared durable state, consistency models and concurrency control mechanisms are essential to prevent corruption.

• Strong Consistency: Guarantees any read returns the most recent write. Often uses locking or consensus protocols like Raft.
• Eventual Consistency: Allows temporary divergence, with replicas converging to the same value over time. Uses Conflict-Free Replicated Data Types (CRDTs).
• Conflict Resolution: Required for concurrent updates. Strategies include last-write-wins, manual merge, or operational transformation.

Durable state systems often implement versioning to track changes over time, enabling audit trails, rollbacks, and debugging.

• Version Identifiers: Each state snapshot gets a unique ID (e.g., timestamp, sequence number, hash).
• State Rollback: Allows reverting to a previous version after an error, relying on checkpointed or versioned state.
• Event Sourcing: An advanced pattern where the state is derived from an immutable, append-only log of all events (changes), providing a complete history.

Durable state is not necessarily permanent. Effective systems include policies for state garbage collection to manage storage costs and privacy.

• Time-To-Live (TTL): Automatically expires state after a defined period.
• Retention Policies: Rules for archiving or deleting state based on age, size, or business logic.
• Compaction: Process of merging incremental updates or deleting obsolete versions to reclaim space, common in log-based storage.

A fault-tolerance technique where an agent's state is periodically saved to stable storage, creating a known-good recovery point. If a failure occurs, the agent can be restarted from the last checkpoint, avoiding recomputation from the very beginning. This is critical for long-running workflows and stateful stream processing. Checkpoints can be:

• Full: Saves the complete state, simpler but larger.
• Incremental/Differential: Saves only changes since the last checkpoint, more efficient.

The reverse process of deserialization and initialization, where a persisted or serialized state is loaded into an active agent's memory, restoring its full operational context. This turns a static data blob back into a live, executable object. Hydration logic must handle:

• Version mismatches between serialized data and current code.
• Re-initializing connections to external resources (APIs, databases).
• Reconstituting complex object graphs and references.

A core durability mechanism where all intended state modifications are first recorded as entries in a sequential, append-only log on stable storage. The actual state (e.g., in a database) is updated only after the log entry is secured. This guarantees recoverability: after a crash, the system can replay the log to reconstruct the last consistent state. WAL is fundamental to databases (PostgreSQL, SQLite) and stateful stream processors (Apache Flink, Kafka Streams).

An architectural pattern where the system's state is not stored directly. Instead, the sequence of all state-changing events is stored as the immutable system of record. The current durable state is derived by replaying these events. This provides a complete audit trail, enables temporal queries ("what was the state at time X?"), and simplifies rebuilding state views. It's often paired with CQRS (Command Query Responsibility Segregation) to optimize read performance.