Multi-Turn Persona Consistency Audit Prompt

This prompt is designed for conversational AI teams who need to verify that an assistant's assigned persona—its tone, role boundaries, vocabulary, and behavioral style—remains stable across an entire multi-turn conversation. The core job-to-be-done is automated quality assurance: you have a system prompt that defines a persona (e.g., a 'helpful senior engineer,' a 'cautious financial advisor,' or a 'concise support agent'), and you need to catch every turn where the model slips out of character, adopts an inconsistent voice, or violates its role constraints. This is not a prompt for evaluating factual correctness, safety policy adherence, or goal completion. It is specifically scoped to persona drift detection.

The ideal user is an AI engineer, evaluation lead, or QA specialist who owns a chat product with a defined persona contract. You should use this prompt when you have: (1) a written persona specification or system prompt that defines the assistant's role, tone, and behavioral boundaries; (2) a multi-turn conversation log where each turn includes the user message and the assistant response; and (3) a need for per-turn scores that flag exactly where persona violations occur. The prompt expects you to supply the persona definition, the full conversation transcript, and optionally a set of persona dimensions to evaluate (e.g., tone, vocabulary, role adherence, formality). If you lack a clear persona specification, this prompt will produce unreliable results because the judge has no stable reference point.

Do not use this prompt for single-turn evaluations, for conversations where the persona is intentionally fluid or context-dependent, or for detecting factual hallucinations. It is also inappropriate for evaluating whether the assistant achieved the user's goal—persona consistency and task success are orthogonal. If your assistant's persona is allowed to shift based on user cues (e.g., matching the user's tone), you should define those permissible shifts in the persona specification rather than expecting rigid consistency. For high-stakes deployments where persona violations could cause brand damage or user harm, always pair automated audit results with human review of flagged turns before taking action on the model or prompt.

This prompt template is designed to be copied directly into your evaluation harness. It instructs the model to act as a persona consistency auditor, scoring each assistant turn against a defined persona specification. The template uses square-bracket placeholders for all variable inputs—swap these with your actual data before sending the request. The output is a structured JSON report with per-turn scores, drift alerts, and a session-level summary, making it straightforward to parse in automated pipelines.

textCopy
You are a persona consistency auditor. Your task is to evaluate a multi-turn conversation between a user and an AI assistant. The assistant has been assigned a specific persona, defined by a set of traits, tone guidelines, and role boundaries. For each assistant turn in the conversation, you will assess whether the assistant's response adheres to the assigned persona.

## PERSONA SPECIFICATION
[PERSONA_DEFINITION]

## CONVERSATION LOG
[CONVERSATION_LOG]

## EVALUATION CRITERIA
For each assistant turn, evaluate the following dimensions on a scale of 1 (complete violation) to 5 (perfect adherence):
- **Trait Consistency**: Does the response reflect the assigned personality traits, knowledge boundaries, and background?
- **Tone Adherence**: Does the response maintain the specified tone (formality, warmth, enthusiasm, etc.)?
- **Role Boundary Respect**: Does the assistant stay within its defined role, avoiding disallowed behaviors or topics?

## OUTPUT FORMAT
Return a valid JSON object with the following structure:
{
  "session_id": "string",
  "persona_version": "string",
  "turn_evaluations": [
    {
      "turn_index": 0,
      "assistant_message": "string (truncated to 100 chars)",
      "trait_consistency_score": 0,
      "tone_adherence_score": 0,
      "role_boundary_score": 0,
      "overall_turn_score": 0.0,
      "violations": ["string description of specific violations"],
      "drift_alert": false,
      "drift_description": "string (only if drift_alert is true)"
    }
  ],
  "session_summary": {
    "average_persona_score": 0.0,
    "total_violations": 0,
    "drift_events": 0,
    "critical_failures": false,
    "overall_assessment": "string"
  }
}

## INSTRUCTIONS
1. Evaluate each assistant turn independently against the persona specification.
2. Flag a `drift_alert` as `true` when a turn shows a significant drop in persona adherence compared to the assistant's established pattern in earlier turns.
3. In `drift_description`, explain what changed (e.g., "Assistant shifted from formal to casual tone mid-conversation" or "Assistant contradicted its stated knowledge limitation from Turn 2").
4. Set `critical_failures` to `true` if any turn contains a role boundary violation that could cause user harm, policy breach, or severe misrepresentation.
5. Be specific in `violations`: cite the exact persona trait or rule that was broken and quote the problematic text.
6. If the conversation log is empty or contains no assistant turns, return a valid JSON object with `turn_evaluations` as an empty array and an appropriate `overall_assessment`.

To adapt this template for your own use, replace [PERSONA_DEFINITION] with a structured description of your assistant's intended persona—include traits, tone rules, knowledge boundaries, and explicit prohibitions. Replace [CONVERSATION_LOG] with the full conversation transcript, ideally with clear speaker labels and turn indices. If your evaluation dimensions differ, modify the criteria list and the corresponding JSON output schema. For high-stakes deployments, add a [RISK_LEVEL] parameter that adjusts scoring thresholds and requires human review when critical_failures is true. Always validate the output JSON against your expected schema before ingesting it into dashboards or automated gates.

This prompt is designed to run as a batch evaluation step, not a real-time guardrail. Feed it a complete conversation transcript, the assigned persona definition, and a list of persona traits to monitor. The model returns per-turn adherence scores and drift alerts. Wire this into your existing evaluation framework (e.g., Braintrust, LangSmith, Arize, or a custom Python pipeline) so that every new prompt or model version is audited against a golden set of conversations before release.

Input assembly requires three structured fields: [CONVERSATION_TRANSCRIPT] as a list of turn objects with turn_index, speaker, and text; [PERSONA_DEFINITION] as a string describing the assistant's assigned role, tone, and behavioral boundaries; and [PERSONA_TRAITS] as a list of specific, observable traits to score per turn (e.g., ["formal tone", "avoids speculation", "uses product terminology correctly"]). The [OUTPUT_SCHEMA] placeholder should be replaced with a strict JSON schema requiring per_turn_scores (array of objects with turn_index, trait_scores, and overall_adherence), drift_alerts (array of objects with turn_index, trait, drift_description, and severity), and session_summary (object with average_adherence and critical_failures). Validate the output against this schema before accepting results. If the model returns malformed JSON, use a retry with schema error feedback rather than silently falling back to a partial score.

Model choice matters. Use a model with strong instruction-following and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro). Avoid smaller or older models that struggle with multi-turn consistency scoring. Set temperature=0 for deterministic scoring. For high-stakes audits—such as pre-release persona validation for a customer-facing assistant—run the prompt three times and take the median score per turn to reduce judge variance. Log every run with the prompt version, model identifier, and raw output for auditability. If a drift alert fires with severity: "critical", route the conversation to a human reviewer before accepting the automated score. Do not auto-remediate persona drift based solely on this prompt's output; it is a detection tool, not a correction mechanism.

Failure modes to instrument: (1) The model conflates persona adherence with factual correctness—monitor for drift alerts that cite factual errors rather than tone or role violations. (2) Long conversations exceed context windows—chunk transcripts into overlapping windows of 20 turns and aggregate scores, flagging turns that appear in multiple windows with divergent scores. (3) The persona definition is too vague to score reliably—if inter-run variance exceeds 0.3 on any trait, refine the trait description before trusting the audit. (4) The model fails to detect gradual drift because it scores each turn independently—add a [DRIFT_WINDOW] parameter (default 5 turns) and instruct the model to compare each turn against the preceding window, not just the persona definition. Next step: run this prompt against a labeled dataset where you've manually annotated persona breaks, then compute precision and recall on drift alerts before integrating into your CI/CD evaluation gate.