Change Data Capture (CDC)

A distributed computing framework designed to ingest, process, and analyze continuous, high-velocity data streams in real-time rather than processing static batches. In fraud detection, stream processors consume CDC event streams to compute sliding window aggregations and velocity checks on transaction attributes. Popular implementations include Apache Kafka Streams, Apache Flink, and Apache Spark Structured Streaming. These engines maintain stateful computations—such as counting distinct credit cards per IP address over a 5-minute window—and emit alerts when thresholds are breached. They guarantee exactly-once semantics through checkpointing and write-ahead logs, ensuring no transaction is double-counted or missed during system failures.

A centralized data management layer that standardizes the storage, transformation, and serving of feature data for both online inference and offline model training. CDC populates the feature store by capturing changes from operational databases and materializing them into low-latency serving layers. Key capabilities include:

• Point-in-time correctness: Ensures training datasets don't leak future information
• Training-serving skew prevention: Identical transformation logic in training and inference paths
• Online serving: Sub-millisecond feature retrieval via Redis or DynamoDB for real-time scoring In fraud systems, features like user_avg_transaction_7d or merchant_chargeback_ratio are pre-computed and served instantly during authorization.

A continuous computation technique that calculates metrics over a dynamically updating, overlapping time interval to provide rolling statistics on streaming data. Unlike fixed tumbling windows, sliding windows update with every new event. In CDC-driven fraud pipelines, sliding windows power velocity checks:

• Count of transactions per card in the last 10 minutes
• Sum of transaction amounts per merchant in the last hour
• Distinct IP addresses per user session in the last 30 minutes These aggregations are maintained in state stores backed by RocksDB, with watermarks handling late-arriving data. The window size directly impacts fraud detection latency—shorter windows catch fast attacks, while longer windows establish behavioral baselines.

A distributed event streaming platform that serves as the central nervous system for real-time fraud detection pipelines. Kafka decouples CDC producers from downstream consumers, providing:

• Durable persistence: Log compaction retains the latest state for each database row, enabling crash recovery
• Horizontal scalability: Partitions distribute CDC event throughput across multiple brokers
• Exactly-once semantics: Idempotent producers and transactional APIs prevent duplicate processing In a typical fraud architecture, Debezium streams MySQL binlog changes into Kafka topics. The fraud scoring engine consumes these topics to update feature store values and trigger complex event processing rules. Kafka Connect sinks materialize enriched events into serving layers like Redis for sub-millisecond lookups during authorization.

The process of augmenting a raw transaction event in real-time with additional contextual data from external services. CDC enables enrichment by maintaining up-to-date materialized views of reference data. Common enrichment sources in fraud pipelines:

• Device fingerprinting: Browser attributes and OS configuration linked via CDC-updated session stores
• Geolocation: IP-to-location lookups joined against streaming transaction events
• Watchlist status: Sanctions and PEP lists materialized from batch updates via CDC
• Merchant risk profiles: Category codes and chargeback histories served from CDC-populated caches Enrichment typically uses stream-table joins, where the transaction stream is joined against a CDC-maintained table of merchant profiles or device histories stored in local state. This contextual data dramatically improves model accuracy by providing signals beyond raw transaction amounts.

The process of generating model predictions in real-time against a single transaction as it arrives, typically within a strict P99 latency budget of 50-100ms to avoid impacting the authorization flow. CDC supports online inference by ensuring feature vectors reflect the most recent state:

• Real-time features: Velocity counts and recent behavioral patterns updated via CDC streams
• Batch features: Historical aggregates refreshed hourly and served from CDC-materialized caches
• Contextual features: Device fingerprints and geolocation enriched at inference time The inference pipeline orchestrates feature retrieval from the online store, model scoring via a REST or gRPC endpoint, and decision output to the rules engine. Shadow mode deployment allows new models to be tested against live CDC traffic without affecting actual authorization decisions, enabling safe validation before promotion.