OpenTelemetry

The OpenTelemetry API provides the instrumentation interface that developers use in their application code to create Spans, Metrics, and Logs. It is language-specific but follows a consistent specification. The OpenTelemetry SDK is the language-specific implementation of the API that handles the processing, sampling, and exporting of telemetry data. Key SDK components include:

• TracerProvider: The factory and manager for Tracers.
• MeterProvider: The factory and manager for Meters.
• Context Propagation: Manages the distributed trace context across service boundaries.
• Samplers: Decide which traces to record to manage volume and cost.

The OpenTelemetry Collector is a vendor-agnostic proxy that receives, processes, and exports telemetry data. It decouples instrumentation from backend analysis, providing a single point for data management. Its modular architecture consists of:

• Receivers: Ingest data in various formats (OTLP, Jaeger, Prometheus, etc.).
• Processors: Filter, batch, transform, or enrich data (e.g., add attributes, tail sampling).
• Exporters: Send data to one or more backends (e.g., Jaeger, Prometheus, Datadog, Splunk).
• Extensions: Provide ancillary functionality like health monitoring. It can be deployed as an agent (per host) or as a gateway (centralized cluster).

The OpenTelemetry Protocol (OTLP) is the primary, vendor-neutral wire protocol for transmitting telemetry data. It defines the gRPC and HTTP/JSON interfaces for sending traces, metrics, and logs from SDKs and Collectors. Key characteristics:

• Efficient: Uses Protocol Buffers (Protobuf) for serialization.
• Reliable: Supports configurable retry logic and queuing.
• Standardized: Provides a single, well-defined protocol, reducing the need for vendor-specific exporters. OTLP is the recommended protocol for all telemetry transmission within the OpenTelemetry ecosystem, replacing legacy formats like Jaeger Thrift or Zipkin JSON.

OpenTelemetry Semantic Conventions are standardized names (attributes) for common telemetry signals. They ensure consistency and interoperability across different services, teams, and vendors by defining what to instrument and how to label it. Major conventions include:

• HTTP Attributes: http.method, http.route, http.status_code.
• Database Attributes: db.system, db.operation, db.statement.
• RPC Attributes: rpc.system, rpc.service, rpc.method.
• Error Attributes: error.type, error.message. Using these conventions enables powerful, out-of-the-box aggregation and filtering in observability backends.

OpenTelemetry Instrumentation Libraries are language-specific packages that automatically generate telemetry for popular frameworks and libraries. They use mechanisms like monkey-patching or wrapper proxies to inject tracing and metrics without requiring manual code changes. Examples include:

• Automatic Instrumentation for Python: Injects tracing into Flask, Django, and requests.
• Java Agent JAR: A Java agent that instruments JDBC, JMS, and HTTP clients at runtime.
• Node.js Instrumentation Packages: Wrappers for express, pg, and redis. This dramatically reduces the effort required to adopt observability and ensures consistent, high-quality data collection.

The OpenTelemetry Specification is the authoritative, language-agnostic document that defines the cross-cutting requirements, concepts, data models, and APIs for the project. It ensures consistency across all language implementations. Core areas it defines:

• Trace Signal: Span lifecycle, links, events, and status.
• Metric Signal: Instrument types (Counter, Gauge, Histogram), aggregation temporality.
• Log Signal: Log record structure and severity mapping.
• Context and Propagation: The W3C TraceContext standard for distributed tracing.
• Resource: Description of the source of telemetry (service name, version, instance ID). The specification is governed by the OpenTelemetry Technical Committee (TC).