Publish-Subscribe Topic Flow

Message Throughput measures the volume of messages published to and consumed from a topic per unit of time (e.g., messages per second). It is a primary indicator of system load and capacity.

• Publish Rate: The rate at which agents produce events.
• Consumption Rate: The rate at which subscribing agents process messages.
• A significant and sustained gap between publish and consumption rates indicates a processing backlog, signaling that subscribers cannot keep pace with publishers, which can lead to increased latency and potential message loss.

End-to-End Latency is the total time elapsed from when a message is published to a topic until it is fully processed by a subscribing agent. This is the user-perceivable delay for event-driven actions.

• Publishing Latency: Time for the broker to acknowledge the published message.
• Delivery Latency: Time for the message to traverse the broker and network to the subscriber.
• Processing Latency: Time for the subscriber agent to execute its logic upon receiving the message.
• Monitoring the 95th and 99th percentile (p95, p99) of this latency is essential for identifying tail latencies that degrade system responsiveness.

Subscription Lag (or consumer lag) quantifies the delay, typically in number of messages or time, between the most recent message published to a topic and the last message successfully processed by a specific subscriber. It is a direct measure of real-time processing health.

• Growing Lag: Indicates a subscriber is falling behind, often due to insufficient resources, blocking operations, or downstream failures.
• Zero Lag: The ideal state, where the subscriber is processing messages as fast as they are published.
• This metric is crucial for SLO adherence in event-driven architectures where timely processing is a business requirement.

This metric tracks the rate at which message processing fails and messages are routed to a Dead-Letter Queue (DLQ). A DLQ is a holding topic for messages that cannot be processed after repeated retries.

• Processing Errors: Failures due to malformed data, business logic exceptions, or unavailable dependencies.
• Poison Pill Messages: Messages that consistently cause subscriber crashes.
• A rising error rate is a key signal for anomaly detection, prompting investigation into subscriber health, data schema changes, or upstream data quality issues.

Fan-Out measures the average number of subscriber agents that receive each published message. Routing Efficiency assesses whether messages are being delivered only to interested, active subscribers.

• High Fan-Out: A message is relevant to many subscribers, typical for broadcast-style events (e.g., system configuration updates).
• Low Fan-Out: Messages are highly targeted, common in workload delegation or direct agent-to-agent communication.
• Inefficient routing, where messages are sent to subscribers that filter them out, wastes network and computational resources. Monitoring this helps optimize topic granularity and subscription filters.

Topic Saturation refers to the utilization of allocated resources (e.g., memory, disk) for a topic. Backlog Depth is the absolute number of messages awaiting consumption across all subscriptions.

• Resource Metrics: Memory used, disk I/O, and partition/segment count for partitioned topics.
• Backlog Analysis: A deep and persistent backlog is a critical alert, indicating systemic overload or a stalled consumer group.
• These metrics are vital for capacity planning and auto-scaling decisions, ensuring the messaging infrastructure can handle peak loads without degradation.

An Agent Interaction Graph is a network model that visualizes communication pathways between agents. In a pub/sub context, it maps publishers to topics and subscribers to those topics, creating a clear topology of the message flow.

• Nodes represent individual agents or topics.
• Edges represent subscription relationships and message flows.
• Used to identify bottlenecks, single points of failure, and understand the overall communication architecture.

A Peer-to-Peer Message Log is a granular record of every direct communication event between agents. For pub/sub, this logs each publish and delivery event.

• Captures sender ID, receiver ID (or topic), message payload (or hash), timestamp, and delivery status.
• Essential for debugging missed messages, auditing agent actions, and reconstructing the sequence of events leading to a system state.
• Differs from a simple topic flow metric by providing full message-level traceability.

Inter-Agent Latency is the critical performance metric measuring the time delay from when one agent publishes a message to a topic until a subscribing agent begins processing it.

• Breaks down into publish latency (agent to message broker), broker processing latency, and delivery latency (broker to subscriber).
• Directly impacts the responsiveness and synchronization of a multi-agent system.
• Monitoring this latency per-topic is key to identifying degraded communication channels.

A Distributed Agent Trace is an end-to-end observability record that follows a single request or unit of work as it propagates through multiple agents via pub/sub and other channels.

• Correlates activities across agent boundaries using a shared trace ID.
• Unifies individual Multi-Agent Spans (an agent's internal processing) with pub/sub message hops.
• Provides a holistic view of causality and total workflow latency, crucial for diagnosing performance issues in complex, event-driven agent workflows.

Coordination Overhead is the aggregate resource cost incurred by agents to communicate and synchronize, as opposed to performing primary task work. In pub/sub systems, this overhead is significant.

• Includes CPU/memory for serializing/deserializing messages, network bandwidth for message transport, and latency spent waiting for messages.
• Measured by tracking the ratio of coordination messages to task-completion messages, or the percentage of agent runtime spent in communication.
• High overhead indicates an inefficient communication architecture or overly chatty agents.

A Multi-Agent SLO (Service Level Objective) is a reliability or performance target defined for a system of interacting agents. For pub/sub topic flows, specific SLOs must be established.

• Examples include: 99.9% of messages delivered to all subscribers under 100ms, or < 0.1% message loss rate per topic per hour.
• Service Level Indicators (SLIs) for pub/sub include message publish rate, end-to-end latency percentiles, subscription backlog depth, and error rate per topic.
• These SLOs ensure the messaging backbone meets the reliability requirements of the business logic running on top of it.