Multi-Agent Based Automation of Dermatology Teledermatology Triage

By analyzing patient-submitted images and history against dermatological ontologies, the system flags suspected melanomas and high-risk lesions for same-day review. This reduces the time-to-diagnosis for urgent cases from days to hours, improving clinical outcomes and patient satisfaction. The throughput gain comes from filtering 60-70% of routine cases from the urgent queue, allowing dermatologists to focus their expertise where it's most needed.

The workflow automates the manual, repetitive tasks of request logging, history intake, and initial image sorting typically handled by nurses or coordinators. By embedding this logic into autonomous agents, it reduces administrative FTE requirements per consultation volume. The savings are realized through higher clinician productivity (more consults per dermatologist) and lower support staff costs, directly improving the service line's operating margin.

Every triage decision is logged with the supporting visual features, patient data points, and guideline references used by the analysis agents. This creates a defensible, searchable audit trail for quality reviews, payer audits, or legal discovery. The system's explainability layer is critical for compliance, demonstrating that triage logic is consistent, evidence-based, and auditable—turning a black-box AI decision into a risk-managed clinical tool.

By providing immediate, automated triage and routing—sending routine cases to teledermatology pools and urgent cases to specialists—the system retains patients within the network. It eliminates the manual back-and-forth that often leads patients to seek care elsewhere. This improves referral capture rates, optimizes network utilization, and creates a seamless access point that supports health system growth and patient retention strategies.

Built on an agentic framework like LangGraph, the workflow orchestrates specialized agents for image analysis (vision models), history intake (LLMs), and routing logic. It integrates via HL7/FHIR APIs with the EHR (e.g., Epic) for patient context and to write back triage notes. A human-in-the-loop dashboard allows dermatologist oversight of uncertain cases. Deployment is typically phased, starting with a pilot on non-melanoma cases to validate safety and logic before full rollout.

Operational control requires a continuous validation loop. The system monitors triage accuracy against gold-standard dermatologist reviews, tracking metrics like false-negative rates for malignancy. A governance committee reviews performance dashboards and approves any changes to agent logic or risk thresholds. This closed-loop monitoring ensures the automation remains safe, effective, and compliant as clinical guidelines and patient populations evolve.

Role: Ingests and structures the initial patient submission (images, history form, chief complaint). Extracts key clinical descriptors (lesion location, duration, symptoms) and patient demographics. Validates data completeness and routes incomplete submissions back to the patient portal.

Implementation: Combines vision models for image metadata extraction (e.g., body location, number of lesions) with an LLM agent to parse free-text history. Outputs a structured JSON case object for the triage pipeline, logging all source data for audit.

Role: Performs primary image analysis using a specialized dermatology vision model. Classifies lesions (e.g., nevus, BCC, SCC, melanoma-suspect) and extracts visual features (asymmetry, border, color, diameter). Outputs a preliminary risk score (e.g., low, medium, high) and a confidence level with supporting visual evidence tags.

Implementation: Runs inference on a fine-tuned model (e.g., ResNet, ViT) hosted in a secure, HIPAA-compliant cloud or on-premise GPU cluster. All model outputs, including confidence scores and feature maps, are stored in an immutable audit log.

Role: Synthesizes the structured history from the Intake Agent with the visual findings from the Triage Agent. Applies clinical guidelines (e.g., ABCDE criteria, Ugly Duckling sign) to generate a consolidated triage recommendation: 'Routine,' 'Urgent (<48h),' or 'Emergency.' Drafts a concise clinical rationale citing the specific patient factors and guidelines used.

Implementation: An LLM agent orchestrated via LangGraph, equipped with a retrieval-augmented generation (RAG) system containing the latest clinical guidelines. Its reasoning chain is captured step-by-step in the audit trail.

Role: Executes the triage decision. For urgent cases, it checks dermatologist calendars in the practice management system (e.g., Epic, Cerner) and proposes the earliest available slot, sending a priority notification. For routine cases, it places them in the standard scheduling queue. For all cases, it triggers patient communication (SMS/portal) with next steps.

Implementation: A rules-based agent with API integrations to the PMS and patient communication platforms. It logs all routing decisions and communication attempts, ensuring no case is dropped and creating a clear operational audit trail.

Role: The system's built-in governance layer. Continuously monitors the workflow, generating an explainability report for each case that links the final decision back to source data, model outputs, and guideline references. Flags cases where agent confidence is low or guidelines conflict for human review. Prepares aggregate reports for quality assurance and regulatory audits.

Implementation: A passive monitoring agent that subscribes to the event stream of the other agents. It constructs a permanent case dossier, essential for defending triage decisions to medical boards, liability carriers, and during Joint Commission surveys.

Role: The mandatory oversight interface for dermatologists and administrators. Presents flagged cases, low-confidence analyses, and a sample of routine cases for quality spot-checking. Allows override of any agent decision with a required text rationale, which is then fed back into the system to improve future agent performance.

Implementation: A secure web dashboard built with a framework like Streamlit or React. It integrates directly with the agent orchestration layer (e.g., LangGraph checkpoints), allowing for seamless intervention without breaking the pipeline. All overrides are versioned and stored as part of the compliance record.

Core bidirectional sync with Epic, Cerner, or Athenahealth to ingest patient history, allergies, and medications for context, and to write back triage decisions, priority flags, and draft clinical notes. This integration uses HL7 FHIR APIs and webhooks to trigger the workflow upon a new consultation request, ensuring the patient record is the single source of truth. A middleware layer handles data normalization and consent checks before any AI processing.

Secure DICOM integration with the hospital's Picture Archiving and Communication System (PACS) or Vendor Neutral Archive (VNA) to retrieve patient-submitted lesion images. The connector standardizes formats, strips protected health information (PHI) for the analysis pipeline, and maintains a strict audit log linking the original image to the AI-generated findings. It must handle various image types (JPEG, PNG from phones) and DICOM studies from prior visits.

Central nervous system built on LangGraph or Temporal, coordinating specialized agents: an Image Analysis Agent (vision model for ABCDE criteria), a History Triage Agent (LLM analyzing patient symptoms), a Risk Scoring Agent (combining outputs into a unified urgency score), and a Routing Agent (applying business rules to assign to urgent clinic, routine queue, or require more info). The engine enforces sequential approval gates and logs every agent's decision rationale.

Integration with Twilio, SendGrid, or the EHR's patient portal to automate communication. For low-risk cases, it sends confirmation and expected wait time. For cases needing better images, it triggers a guided re-capture request. For urgent referrals, it can initiate secure messaging to the on-call dermatologist's HIPAA-compliant platform. All outbound messages are templated, logged, and include no PHI in subject lines.

Immutable audit trail system that captures the complete chain of evidence: raw inputs, each agent's output, confidence scores, the final triage decision, and the specific business rule or guideline (e.g., NCCN melanoma guidelines) invoked. This is stored separately from operational databases and is queryable for compliance reviews, liability defense, and model performance monitoring. It feeds dashboards for QA and regulatory reporting.

Direct API integration with scheduling systems like Epic Cadence or OpenSlot. For high-priority cases, the workflow checks real-time provider availability and can place a tentative "hold" on an urgent slot, generating a task for staff to confirm. For routine cases, it appends the patient to a specific provider's backlog queue. This closes the loop from triage to appointment, preventing manual re-entry and reducing time-to-treatment.