Causal Convolution

The defining mechanism of a causal convolution is an asymmetric masking of the input sequence. During the convolution operation, the filter is explicitly prohibited from accessing any element at time step t+1 or later.

• Implementation: Achieved by padding only the left side of the input sequence and shifting the kernel output.
• Contrast with standard convolutions: A standard 1D convolution with 'same' padding would center the kernel, causing the output at time t to be influenced by future inputs t+1 and t-1.
• Result: The output sequence is perfectly aligned with the input sequence in time, guaranteeing that the model cannot cheat by peeking into the future of a transaction stream.

Unlike recurrent neural networks (RNNs) which must process a sequence step-by-step, causal convolutions enable fully parallel computation across the entire time dimension during training.

• Mechanism: Because the receptive field is fixed and known in advance, the convolution can be computed as a single matrix multiplication over the entire padded input sequence.
• Efficiency gain: This eliminates the sequential bottleneck of backpropagation through time (BPTT), drastically reducing training wall-clock time on GPUs.
• Production impact: Models can be retrained on massive historical transaction logs much faster, enabling more frequent updates to adapt to new fraud patterns.

A single causal convolution layer has a very limited view of the past. To capture long-term dependencies, causal convolutions are combined with dilation factors in a Temporal Convolutional Network (TCN).

• Dilation mechanism: Dilation inserts gaps between kernel elements, allowing the filter to operate on a wider input area without increasing the number of parameters.
• Exponential growth: By stacking layers with dilation factors that double each time (1, 2, 4, 8, ...), the model's receptive field grows exponentially with network depth.
• Example: A TCN with 8 layers and a kernel size of 3 can cover over 255 past transactions, enabling the detection of slow-burn fraud schemes that unfold over hundreds of events.

Causal convolutions provide a direct, fixed-length path for gradient flow, mitigating two critical failure modes of recurrent architectures.

• Vanishing gradients: In RNNs, gradients must traverse a long chain of repeated multiplications, causing them to vanish. A TCN's gradient path length is proportional to network depth, not sequence length.
• State explosion: RNNs maintain a hidden state that can grow unstable over very long sequences. Causal convolutions have no recurrent state; the output is a pure function of the bounded receptive field.
• Training stability: This architectural choice leads to more stable and predictable training dynamics when modeling user transaction histories with hundreds of steps.

For real-time fraud scoring, a causal convolution model can be deployed as a streaming processor that emits a prediction for each new transaction with minimal overhead.

• No recomputation: Unlike a Transformer that must recompute attention over the entire sequence, a causal CNN only needs to apply the convolutional kernels to the most recent input window.
• Fixed latency: The inference time per transaction is constant, determined by the kernel size and network depth, not the total length of the user's history.
• Use case: This property is critical for pre-authorization fraud checks where a decision must be returned in under 50 milliseconds to avoid impacting the customer checkout experience.

The causal constraint provides a formal mathematical guarantee that the model respects the arrow of time, which is critical for regulatory compliance and model explainability.

• Formal definition: For an input sequence x = [x_0, x_1, ..., x_T], the output y_t is a function f(x_0, x_1, ..., x_t) only.
• Auditability: When a transaction is flagged as fraudulent, the model's reasoning is confined to events that occurred before the alert. This prevents the logically incoherent situation of a future transaction 'causing' a past alert.
• Regulatory alignment: This property aligns with financial auditing standards that require decisions to be based solely on information available at the time of the event.

A neural network architecture that combines causal convolutions with dilated convolutions to model sequential data. Unlike RNNs, TCNs process entire sequences in parallel, offering faster training. The dilation factor increases exponentially with depth, giving the network an exponentially large receptive field to capture long-range transaction patterns without violating temporal order.

• Key advantage: Parallelizable computation over time steps
• Core mechanism: 1D fully-convolutional architecture with zero padding of length (kernel_size - 1) applied only to the left
• Use case: Modeling years of transaction history to detect slow-burn account takeover

A convolution operation where the kernel is applied over an area larger than its length by skipping input values with a defined dilation factor. When combined with causal convolution, it allows the network's receptive field to grow exponentially with depth while preserving temporal causality.

• Formula: receptive_field = 1 + (kernel_size - 1) * sum(dilation_rates)
• Common pattern: Dilation rates double at each layer: 1, 2, 4, 8, ...
• Benefit: Captures very long transaction sequences without requiring a proportionally deep network or large kernels

The extent of input sequence that influences a single output neuron. In a causal TCN, the receptive field determines how far back in transaction history the model can look to make a prediction at time step t. A larger receptive field captures long-range dependencies like seasonal fraud patterns.

• Calculation: Determined by kernel size, number of layers, and dilation factors
• Design trade-off: Larger receptive fields require more layers or larger dilations, increasing model capacity and memory
• Fraud context: A receptive field of 100 time steps might capture a fraudster's slow probing behavior over several days

A sequence model that replaces recurrence and convolution entirely with self-attention. Unlike causal convolution, which has a fixed local receptive field, self-attention allows every output to attend directly to all previous inputs with O(1) path length. Positional encoding is required to inject temporal order.

• Contrast with causal convolution: Global receptive field from the first layer vs. growing receptive field with depth
• Complexity: O(n²) memory for self-attention vs. O(n) for convolution
• Fraud application: Detecting relationships between a current transaction and any past anomalous event, regardless of distance

The gradient-based learning algorithm for recurrent networks, where the computation graph is unrolled over the temporal dimension. Gradients flow backward through each time step. Causal convolution avoids BPTT entirely, using standard backpropagation through the spatial layers, which is more stable and parallelizable.

• RNN challenge: Vanishing or exploding gradients over long unrolled sequences
• Causal convolution advantage: Gradients flow through a fixed-depth network, not across time steps
• Training stability: Convolutional architectures typically converge faster and more reliably on long sequences

A structured state space sequence model that offers a linear-time alternative to both causal convolution and self-attention. Mamba uses a selective scan mechanism that compresses context into a hidden state, efficiently handling sequences with hundreds of thousands of time steps.

• Complexity: O(n) time and memory, matching causal convolution's efficiency
• Key innovation: Input-dependent parameters allow the model to selectively propagate or forget information
• Fraud relevance: Processing ultra-long transaction histories (100k+ events) without truncation or summarization