Critical User Journey (CUJ)

A CUJ is defined from the user's perspective, not the system's architecture. It represents a complete, end-to-end task that delivers core value. For an e-commerce site, this is "search for product, add to cart, complete checkout." For an AI chatbot, it's "ask a complex question, receive a grounded, accurate answer." The journey's success is directly tied to key business outcomes like revenue, conversion, or user retention. Defining SLOs on CUJs ensures engineering efforts protect what matters most to users and the business.

Every step in a CUJ must be quantifiable using Service Level Indicators (SLIs). These are the raw metrics that indicate health and performance. For AI services, relevant SLIs include:

• Model Inference Latency: Total time for request-to-response.
• Time To First Token (TTFT): Responsiveness for streaming outputs.
• Error Rate: Percentage of failed requests (e.g., 5xx errors, model crashes).
• Task Success Rate: For agents, the percentage of journeys completed without intervention.
• Answer Faithfulness Score: For RAG, the proportion of the answer supported by source context. These SLIs provide the data to evaluate whether the CUJ is meeting its Service Level Objective (SLO).

The CUJ's SLIs are used to set a Service Level Objective (SLO), a target reliability over a time window (e.g., "99.9% of checkout journeys complete in under 2 seconds per page over 30 days"). The difference between 100% and the SLO is the Error Budget—the allowable unreliability. This budget becomes a central management tool:

• Burn Rate: Measures how quickly the budget is consumed.
• Release Gating: New features can be deployed if they don't risk exhausting the budget.
• Priority Setting: Focuses engineering effort on fixes that protect the CUJ. This creates a feedback loop where user experience directly governs engineering priorities.

A single CUJ typically depends on multiple backend services, databases, and third-party APIs. For an AI-powered journey like "get a summarized answer from a document," dependencies include:

• Vector Database for semantic retrieval.
• Inference Endpoint for the LLM.
• Authentication Service.
• External Data APIs. The overall CUJ SLO is a Composite SLO derived from the individual SLOs of its components. This highlights systemic fragility through Tail Latency Amplification, where the slowest dependency (e.g., p99 database query) dictates the user's p99 experience.

Instrumenting CUJs transforms monitoring from reactive to proactive. Instead of watching server CPU, teams monitor the Golden Signals (latency, traffic, errors, saturation) for the journey itself. This enables:

• Multi-Window Alerting: Triggering alerts based on SLO burn rate across short (1hr) and long (30-day) windows to distinguish blips from sustained degradation.
• Canary Analysis: Deploying new models or code to a fraction of CUJ traffic to validate SLO compliance before full rollout.
• Graceful Degradation: Designing fallback mechanisms (e.g., returning a cached answer) to protect the CUJ's core SLO when a non-critical dependency fails.

For AI services, CUJs must account for non-traditional quality metrics beyond latency and errors. These require specialized SLIs and SLOs:

• Hallucination Rate: Target percentage of factually incorrect generations.
• Retrieval Precision@K: Relevance of top documents fetched for a RAG system.
• Instruction Following Accuracy: Adherence to prompt constraints.
• Data Drift Detection: Monitoring for statistical shifts in input data that degrade model performance.
• Cost per Inference: Balancing quality with infrastructure expenditure via a Cost Efficiency SLO. Defining CUJs forces explicit quantification of these probabilistic quality aspects, moving AI from art to engineered service.

A Service Level Objective (SLO) is a quantitative target for the reliability, performance, or quality of a service, expressed as a percentage of requests that must meet a specific Service Level Indicator (SLI) over a defined time window. For a CUJ, the SLO is the formal, measurable goal derived from user expectations.

• Example: "99.9% of chatbot query responses must have a latency under 200ms."
• SLOs are internal goals, not customer-facing contracts.
• They create a clear, shared target for engineering and product teams.

A Service Level Indicator (SLI) is a directly measurable metric that quantifies a specific aspect of a service's performance, such as latency, error rate, or throughput. SLIs are the raw measurements used to evaluate compliance with an SLO. For AI services, SLIs are often specialized.

• Core SLI Examples: Model inference latency, token throughput, task success rate.
• AI-Specific SLIs: Hallucination rate, retrieval precision, answer faithfulness.
• A CUJ is typically monitored by a composite set of SLIs covering its entire sequence.

A Service Level Agreement (SLA) is a formal contract between a service provider and a customer that defines the minimum expected service level, often including financial penalties or remedies if Service Level Objectives (SLOs) are not met. While an SLO is an internal target, an SLA is an external promise.

• SLAs are typically less aggressive than internal SLOs to provide a safety margin.
• Violating an SLA has business consequences (e.g., service credits).
• CUJs help define which SLOs are critical enough to be included in customer-facing SLAs.

An error budget is the allowable amount of service unreliability, calculated as 100% - SLO. It defines the risk a team can accept for deploying new features or making changes without violating the SLO. It turns reliability from a constraint into a manageable resource.

• Calculation: If the SLO is 99.9% availability, the error budget is 0.1% unreliability.
• Teams can "spend" the budget on risky deployments.
• Once the budget is exhausted, only reliability-focused work (e.g., bug fixes, stability improvements) is permitted until the budget is replenished.

A golden signal is one of four fundamental metrics used in Site Reliability Engineering (SRE) to comprehensively monitor service health: Latency, Traffic, Errors, and Saturation (LTES). These signals provide a first-order understanding of any service's state.

• Latency: Time to service a request (e.g., p95 inference time).
• Traffic: Demand on the system (e.g., queries per second).
• Errors: Rate of failed requests (e.g., 5xx HTTP errors, model failures).
• Saturation: How "full" the service is (e.g., GPU memory utilization, queue depth). Monitoring these for each CUJ provides a complete operational picture.

A canary deployment is a release strategy where a new version of a service is deployed to a small, representative subset of users or traffic. Its performance and stability are monitored against key SLIs before a full rollout. This is a primary method for validating that changes do not break CUJ SLOs.

• Process: Route 5% of CUJ traffic to the new model version, compare its error rate and latency to the baseline.
• Goal: Detect regressions early, before they impact all users and consume the error budget.
• Requires robust A/B testing and experiment tracking infrastructure.