Discovery Protocol

Discovery protocols are categorized by their architectural pattern for storing and querying service metadata.

• Centralized Registry: A single, authoritative source (e.g., a dedicated server or database) holds all service metadata. Clients query this central point. This simplifies management and provides a global view but creates a single point of failure and potential bottleneck. Examples include traditional service registries like Consul or Eureka.

• Decentralized (Peer-to-Peer): Service metadata is distributed across the network nodes themselves. Nodes broadcast advertisements and respond to queries directly. This eliminates a central point of failure and scales organically but can be more complex to manage and may have eventual consistency. Multicast DNS (mDNS) and DNS-Based Service Discovery (DNS-SD) are classic examples of decentralized protocols.

Protocols define how service information flows between providers and consumers.

• Pull (Client-Driven): The discovery client actively queries a registry or network to find services. This is often done via polling at regular intervals. It's simple to implement but introduces latency between a service becoming available and being discovered, and consumes bandwidth even when no changes occur.

• Push (Event-Driven): The registry or service providers actively notify subscribed clients when changes occur (e.g., a new service registers, or a health check fails). This enables near real-time discovery and is more efficient but requires a subscription mechanism and more complex client-side logic to handle events. Many modern registries (e.g., Consul with its watch feature) support a hybrid or push-like model.

A core function is defining what information is advertised about a service beyond its network location.

• Basic Metadata: Includes the service name, version, network endpoint (IP/port), and protocol (HTTP, gRPC).

• Extended Metadata: For AI tool discovery, this becomes critical. Advertised data includes:

  • Tool schemas (input/output parameters, data types via OpenAPI/JSON Schema).
  • Capability descriptions and authorization scopes.
  • Health status and load metrics.
  • Semantic tags (e.g., database, payment, llm) for tag-based discovery.

This rich metadata enables schema-based discovery, where an AI agent can search for tools that accept a specific data structure or perform a specific type of function.

Protocols must manage the entire lifecycle of a service entry.

• Registration: How a service announces itself.

  • Self-registration: The service autonomously registers on startup (common in microservices).
  • Third-party registration: An external agent (like a deployment orchestrator) registers the service.

• Health Checking: Continuous verification that a registered service is functional.

  • Active checks: The registry periodically calls a /health endpoint on the service.
  • Passive checks: The registry infers health from the success/failure of actual service requests.

• Deregistration: Clean removal of a service entry.

  • Explicit deregistration during graceful shutdown.
  • Automatic expiration via lease-based registration, where a service must periodically renew its lease or be removed. This handles ungraceful failures.

Discovery protocols exist at multiple layers of the stack.

• Network Layer (Infrastructure):

  • mDNS/DNS-SD: Used in local networks for discovering printers, speakers, and IoT devices. Apple Bonjour is a major implementation.
  • Simple Service Discovery Protocol (SSDP): Used by UPnP devices.

• Application Layer (Microservices/AI):

  • Consul: Provides service discovery, health checking, and a key-value store. Uses a gossip protocol for cluster membership and RPC for requests.
  • etcd/ZooKeeper: Primarily consistent key-value stores often used as the backing store for a discovery system (e.g., Kubernetes uses etcd).
  • Model Context Protocol (MCP): In AI, MCP servers advertise their available tools and data sources via a standardized introspection endpoint. AI clients (like an IDE agent) discover these capabilities dynamically.

For AI agents, discovery protocols bridge the gap between high-level intent and executable functions.

• Dynamic Binding: The agent's request for an action ("book a flight") is resolved at runtime to a specific tool endpoint discovered via the protocol.

• Capability Negotiation: The agent and discovered tool can exchange information on supported versions or features to ensure compatibility before invocation.

• Tool Resolution: When multiple tools match a query (e.g., multiple database connectors), the system uses rules (version priority, load, tags) to select one.

• Namespace Management: Tools are organized into logical groups (e.g., finance/, customer_support/) to prevent naming collisions and scope queries, which is vital in large enterprise deployments with many AI agents and tools.

The automated process by which client applications dynamically locate network endpoints and metadata for available services within a distributed system. For AI agents, this is the foundational step that precedes tool invocation.

• Key Mechanism: Often implemented via a registry (e.g., Consul, etcd) or a protocol (e.g., DNS-SD).
• Agent Use Case: An agent queries a discovery service to find the current IP and port for a "database-query" tool before making an API call.

A centralized or decentralized catalog that stores metadata, schemas, and endpoint information for executable functions. It is the persistent store that a discovery protocol queries.

• Centralized Model: A single database (e.g., a company-internal tool hub) acting as the source of truth.
• Decentralized Model: A peer-to-peer network where tools advertise themselves, enhancing resilience but adding complexity.
• Stored Metadata: Includes OpenAPI schemas, authentication methods, version, and health status.

A declarative file (JSON/YAML) that describes a tool's capabilities, interface schema, and requirements. It is the source document for registration.

• Contents: Defines the tool's name, description, input_schema (parameters), output_schema, and required authentication scopes.
• Runtime Use: Served by an introspection endpoint or submitted to a registry during the registration protocol handshake.
• Example: A weather.yaml manifest describing a function that takes a location string and returns a temperature float.

The standardized rules and message formats defining how a service announces its availability to a registry. This is how tools populate a discovery system.

• Self-Registration: The tool autonomously calls the registry's API on startup, providing its manifest.
• Lease-Based: Registration expires unless renewed, automatically cleaning up failed services.
• Deregistration: A graceful shutdown signal removes the tool from the registry, preventing stale entries.

The runtime mechanism that connects a client's abstract request to a specific tool implementation, based on discovery results. It decouples the agent's intent from the concrete endpoint.

• Process: 1. Agent decides it needs a "send-email" tool. 2. It queries the discovery protocol. 3. The protocol returns the endpoint for the current v2.1 email service. 4. The agent binds to and calls that specific endpoint.
• Benefit: Allows for seamless version upgrades, A/B testing, and failover without agent code changes.

A dedicated API endpoint (e.g., /.well-known/tools) on a service that returns a structured description of its available functions. It enables dynamic, on-the-fly discovery.

• Function: When queried with a GET request, it returns the tool's manifest or a list of its capabilities.
• Use in Discovery: A registry or agent can poll this endpoint to verify a tool's schema and health.
• Standardization: Critical for declarative tooling where tools are self-describing, reducing integration code.