Process Instance

Each process instance maintains its own private execution context, including:

• Local Variables: Data scoped to the instance (e.g., order_id, processing_status).
• Execution Pointer: The current position within the workflow definition.
• Call Stack: The state of nested or parallel executions. This isolation ensures that one instance's data or failures do not affect others, enabling concurrent, independent processing of multiple business cases (e.g., 10,000 simultaneous loan applications).

Every instance is assigned a globally unique identifier (UUID) at creation and progresses through a defined lifecycle state machine:

• States: RUNNING, SUSPENDED, COMPLETED, FAILED, TERMINATED.
• Lifecycle Events: Creation, start, pause, resume, completion, or manual termination. The engine manages this lifecycle, allowing for operations like pausing a long-running instance for manual review or bulk-terminating instances based on a business rule.

A core reliability feature is the persistent storage of instance state. This involves:

• Checkpointing: Periodically saving the complete runtime state (variables, execution pointer) to a database.
• Fault Tolerance: Enabling the engine to recover and resume execution from the last checkpoint after a system crash.
• Deterministic Replay: The ability to reconstruct the exact execution path from the persisted event history for debugging. This is foundational for long-running workflows that may execute for days or weeks.

An instance generates an immutable audit trail—a chronological log of all events and state changes. This includes:

• Task Execution Logs: Start/end times, inputs, outputs, and errors for each activity.
• State Transitions: Record of each lifecycle state change.
• Decision Points: Logs of evaluated conditions in conditional branching. This trail is critical for compliance (e.g., financial transaction tracing), debugging complex failures, and generating operational metrics like average execution time.

Process instances interact with the outside world through events and messages. Key mechanisms include:

• Correlation IDs: A unique key (e.g., customerId) used to route external events (like a payment confirmation) to the correct waiting instance.
• Event-Driven Triggers: Instances can pause at a step, listening for an external event before proceeding, enabling event-driven orchestration.
• Orchestration API: External systems can start, query, or signal instances via a dedicated API. This allows integration with user interfaces or other backend services.

Instances implement robust error handling patterns to manage failures gracefully:

• Retry Logic: Automatic retry of failed tasks with configurable policies (e.g., exponential backoff).
• Circuit Breakers: Preventing cascading failures by halting calls to a failing service.
• Compensating Transactions: For Saga patterns, if a long-running transaction fails, a series of semantically inverse operations are executed to rollback partial changes, ensuring business process consistency.

The runtime environment that interprets a workflow definition and manages the lifecycle of its process instances. It handles state transitions, task dispatching, and event routing.

• Core Function: Executes the logic defined in a workflow model (e.g., a DAG or state machine).
• Responsibilities: Manages concurrency, persistence, retries, and failure handling for all active instances.
• Example: Apache Airflow's scheduler and executor, Temporal's workflow worker.

The blueprint or template that specifies the structure, tasks, and control flow logic for a class of processes. A process instance is a single execution of this definition.

• Declarative vs. Imperative: Can be defined declaratively (YAML, DSL) or imperatively (code).
• Contains: Task nodes, control flow logic (branches, loops), input/output schemas, and error handlers.
• Analogy: A class in object-oriented programming, where a process instance is an object of that class.

The mechanism by which a workflow engine durably stores and retrieves the runtime state of a process instance, enabling reliability and recovery.

• What is Stored: The execution pointer, variable values, and history of events.
• Purpose: Guarantees execution continuity across system failures, restarts, or host migrations.
• Implementation: Often uses a dedicated database (e.g., PostgreSQL, Cassandra) as the system of record.

A discrete, executable unit of work within a workflow that is invoked by the engine during a process instance's execution.

• Types: Can be a function call, an API request, a database query, or a human task.
• Execution: The workflow engine dispatches activities, often to external workers, and awaits their completion.
• Relationship to Instance: A single process instance will execute a sequence of one or more activities.

An architectural pattern where the state of a process instance is derived from an immutable, append-only log of all state-changing events.

• Core Principle: The event log is the source of truth; the current state is a projection of that log.
• Benefit for Orchestration: Enables deterministic replay, audit trails, and easy debugging by reconstructing any past instance state.
• Contrast: Differs from storing only the current state (CRUD), as it preserves full history.

A critical property for activities and workflows where performing the same operation multiple times produces the same, unchanged result as performing it once.

• Why it Matters: Process instances may retry failed steps or recover from crashes. Idempotency prevents duplicate side effects (e.g., charging a customer twice).
• Achieving It: Using unique idempotency keys, conditional checks, or designing operations to be naturally idempotent (e.g., SET status = 'complete').
• Foundation: Essential for building fault-tolerant orchestration systems.