Amazon S3 (Simple Storage Service)

S3 is engineered for 11 9's (99.999999999%) durability and 99.99% availability over a given year for Standard storage.

• Durability: Protection against data loss. S3 automatically replicates data across a minimum of three physically separated Availability Zones (AZs) within an AWS Region.
• Availability: The proportion of time the service is accessible. S3's multi-AZ design ensures resilience to the failure of an entire data center.
• Storage Classes: Different classes optimize for cost vs. access frequency (e.g., S3 Standard for frequent access, S3 Glacier for archival), all while maintaining high durability.

S3 provides two read-after-write consistency models, a critical consideration for application design.

• Strong Consistency: For PUTS of new objects and DELETES, S3 now provides strong read-after-write consistency. After a successful write, any subsequent read request will immediately return the latest data.
• Eventual Consistency: Historically the default model. For overwrites (PUTS to an existing key) and listings, changes are propagated across all AZs and become visible to all clients eventually, typically within milliseconds. Applications must be designed to handle potential temporary stale reads in these cases.

All interactions with S3 occur via a secure, standardized HTTP/REST API. Every operation (GET, PUT, DELETE, LIST) is an HTTP request signed with AWS credentials.

• Authentication & Authorization: Controlled through a combination of:
  • IAM Policies: Attached to AWS users/roles for fine-grained permissions.
  • Bucket Policies: JSON-based resource policies attached directly to the bucket.
  • Access Control Lists (ACLs): A legacy, less granular permission mechanism.
  • Pre-signed URLs: Time-limited URLs granting temporary access to a specific object, useful for secure sharing.
• Encryption: Supports server-side encryption (SSE-S3, SSE-KMS, SSE-C) and client-side encryption for data at rest.

S3 Lifecycle configuration rules automate moving objects between cost-optimized storage classes or expiring them, which is essential for managing agent memory logs and historical data.

• Transition Actions: Automatically move objects to a different storage class after a defined period (e.g., move from S3 Standard to S3 Standard-IA after 30 days, then to S3 Glacier after 90 days).
• Expiration Actions: Permanently delete objects or non-current versions after a set time.
• Key Storage Classes:
  • S3 Standard: General purpose, high durability/availability/frequent access.
  • S3 Intelligent-Tiering: Automatically moves data between access tiers based on changing access patterns.
  • S3 Glacier Instant Retrieval: Archival with millisecond retrieval.
  • S3 Glacier Deep Archive: Lowest-cost storage for data accessed once or twice a year.

A data integrity protocol where all modifications are written to a durable log file before they are applied to the main data structures. This ensures recoverability in case of a crash.

• Analogy: Similar to a database transaction log.
• S3 Parallel: While S3 offers strong consistency for new object PUTs, architecting an agentic system often involves using S3 as the durable destination for checkpoints and state snapshots that are generated after a WAL process in the agent's operational memory.