Envoy Proxy

Envoy's control plane is decoupled from its data plane and is configured dynamically through a set of Discovery Service (xDS) APIs. This allows for real-time updates without restarting proxies. Key APIs include:

• CDS (Cluster Discovery Service): Defines upstream clusters of hosts.
• EDS (Endpoint Discovery Service): Provides fine-grained endpoint (host/port) information for clusters.
• LDS (Listener Discovery Service): Configures network listeners (ports, filters).
• RDS (Route Discovery Service): Manages routing tables for HTTP traffic. This architecture is fundamental to service meshes like Istio, where the control plane (e.g., Istiod) pushes configuration to Envoy sidecars.

Envoy processes network traffic through a modular pipeline of filters. Each connection or request passes through a chain of filters that can inspect, modify, or route traffic. Key filter types include:

• Listener Filters: Operate on raw connections (e.g., TLS inspection).
• Network Filters: Handle L3/L4 TCP/UDP tasks (e.g., rate limiting, MongoDB sniffing).
• HTTP Filters: Operate on HTTP/1.1, HTTP/2, and gRPC streams (e.g., routing, compression, JWT validation). Filters can be written in C++ or, via WebAssembly (Wasm), in other languages, allowing for extensible, sandboxed custom logic.

Envoy provides sophisticated, out-of-the-box load balancing algorithms that go beyond simple round-robin. These are critical for resilience and performance in microservices:

• Weighted Least Request: Routes to the host with the fewest active requests.
• Ring Hash / Maglev: Consistent hashing for session affinity.
• Random: Selects a random healthy host.
• Original Destination: Routes to the original destination address (useful for transparent proxy modes). Load balancing decisions are made per-request and integrate with health checking to automatically exclude unhealthy endpoints.

Envoy generates extensive, structured telemetry data, making distributed systems observable. It exports metrics, logs, and traces through standardized interfaces:

• Statistics (Metrics): Thousands of pre-defined counters, gauges, and histograms for L4 and L7 traffic, accessible via the /stats admin endpoint.
• Distributed Tracing: Native support for OpenTelemetry (OTel), Zipkin, Jaeger, and Datadog, propagating trace headers across service boundaries.
• Access Logs: Detailed, customizable logs for every request, which can be emitted in JSON or plain text to stdout or files. This data is essential for monitoring latency, error rates, and traffic patterns.

Envoy implements several circuit-breaking and failure recovery patterns to prevent cascading failures:

• Outlier Detection: Dynamically ejects hosts from load balancing pools based on consecutive failures (5xx errors, timeouts, TCP failures).
• Retry Policies: Configurable retries for failed requests with budget limits and predicate-based retry conditions.
• Timeouts: Configurable per-route timeouts for connections, requests, and idle periods.
• Circuit Breakers: Limits on concurrent connections and pending requests to upstream clusters. These features allow applications to gracefully degrade when dependencies fail.

Envoy acts as a full-featured TLS termination and initiation proxy, centralizing certificate management and enabling zero-trust security models:

• TLS Termination: Decrypts incoming TLS traffic at the proxy, forwarding plaintext to the local application.
• TLS Origination: Encrypts outbound traffic from the application to upstream services.
• Mutual TLS (mTLS): Validates client certificates for both incoming and outgoing connections, a cornerstone of service mesh security. Envoy can automatically rotate certificates via the Secret Discovery Service (SDS) API, integrating with systems like SPIFFE/SPIRE.

The sidecar pattern is a deployment model where a helper container (the sidecar) is deployed alongside the primary application container in the same pod (Kubernetes) or task. The sidecar extends or enhances the application's functionality without modifying the application itself. Envoy Proxy is deployed as a sidecar to handle all inbound and outbound network traffic for the application, providing a transparent layer for service discovery, traffic routing, and observability. This pattern is foundational to service mesh architectures.

xDS is a family of discovery protocols that Envoy uses to dynamically configure itself. A control plane (like Istio) serves xDS APIs (e.g., CDS-Cluster Discovery, EDS-Endpoint Discovery, LDS-Listener Discovery, RDS-Route Discovery). Key features:

• Dynamic Updates: Envoy fetches configuration updates without restarting.
• Incremental xDS (Delta xDS): Only sends changes, improving efficiency.
• Aggregated Discovery Service (ADS): Allows updates to be delivered on a single gRPC stream for atomic configuration changes. This protocol is central to Envoy's operation in dynamic, cloud-native environments.

Health checking is the mechanism by which Envoy determines the operational status of upstream service endpoints (agents). Envoy performs active health checks by periodically sending HTTP, TCP, or gRPC requests to endpoints. If an endpoint fails consecutive checks, it is removed from the load balancing pool (outlier detection). Passive health checks (outlier detection) eject endpoints based on runtime failure rates (e.g., HTTP 5xx errors, connection timeouts). This is critical for maintaining system reliability in agent orchestration.

Envoy provides sophisticated load balancing algorithms to distribute traffic across a discovered set of healthy upstream endpoints (agents). Key algorithms include:

• Round Robin: Distributes requests sequentially.
• Least Request: Favors endpoints with the fewest active requests.
• Ring Hash / Maglev: Consistent hashing for session affinity.
• Random: Selects a random healthy host.
• Weighted Least Request: Combines least request with configurable endpoint weights. Envoy's load balancing is dynamic, instantly reacting to health check and endpoint discovery (xDS) updates.