LSTM Sequence Model

The cell state acts as a linear highway running through the entire sequence, allowing gradients to flow unchanged across many time steps. This is the core innovation that prevents the vanishing gradient problem that plagues standard RNNs. In SCADA anomaly detection, the CEC preserves the context of a normal operational sequence—such as a multi-step breaker reclosing procedure—across hundreds of individual Modbus or DNP3 commands without losing the signal.

The forget gate uses a sigmoid activation to output values between 0 and 1, determining what proportion of the previous cell state to retain or discard. It examines the current input and the previous hidden state to decide which historical information is no longer relevant. For SCADA sequences, this allows the model to flush stale context—such as a completed maintenance mode flag—so it does not interfere with detecting anomalies in the newly resumed operational state.

This two-part mechanism controls what new information is stored in the cell state:

• The input gate (sigmoid layer) decides which values to update
• The candidate layer (tanh activation) creates a vector of new candidate values Together, they selectively incorporate relevant new observations—such as a sudden voltage sag or an unexpected write command—into the long-term memory without overwriting critical historical context.

The output gate controls what information from the cell state is exposed to the next layer or time step. It applies a sigmoid filter to the concatenated input and previous hidden state, then multiplies it with a tanh-squashed cell state to produce the hidden state. In a SCADA command prediction model, this hidden state encodes the model's prediction of the next expected function code, device address, and value range—flagging any deviation as a potential anomaly.

A bidirectional LSTM (BiLSTM) runs two independent LSTM layers: one forward through the sequence and one backward. The hidden states from both directions are concatenated at each time step, giving the model full context of both preceding and subsequent commands. This is critical for SCADA anomaly detection because a command that appears normal in isolation—such as a breaker open instruction—may only be identifiable as malicious when the subsequent commands reveal an unsafe sequence.

In a seq2seq LSTM configuration, an encoder LSTM compresses the entire input command sequence into a fixed-length context vector, and a decoder LSTM generates the predicted next sequence autoregressively. For OT security, this enables the model to predict not just the next single command but the entire expected sequence of operations—such as a full transformer tap-change procedure—and raise an alert when the actual SCADA traffic diverges from the predicted pattern.

The foundational architecture from which LSTMs evolved. Standard RNNs process sequential data by maintaining a hidden state that captures information about previous inputs. However, they suffer from the vanishing gradient problem, where the influence of earlier inputs decays exponentially during backpropagation through time. This makes them incapable of learning long-term dependencies beyond roughly 10 timesteps. LSTMs were explicitly designed to overcome this limitation through gating mechanisms.

The defining innovation of the LSTM architecture. Three gates regulate information flow:

• Forget gate: Decides what information to discard from the cell state using a sigmoid activation
• Input gate: Determines which new information to store in the cell state
• Output gate: Controls what parts of the cell state are exposed to the hidden state output

These gates enable the network to selectively remember or forget information over hundreds of timesteps, making them ideal for SCADA command sequences where operational context spans long intervals.

The cell state is the horizontal conveyor belt running through the entire LSTM chain. It acts as a memory highway where information can flow unchanged across many timesteps with only minor linear interactions. This design is the direct solution to the vanishing gradient problem. In SCADA anomaly detection, the cell state preserves the context of prior legitimate commands—such as a maintenance mode activation—allowing the model to correctly predict that subsequent atypical commands are expected rather than malicious.

A simplified variant of the LSTM that merges the cell state and hidden state into a single vector and combines the forget and input gates into a single update gate. GRUs have fewer parameters than LSTMs, making them computationally faster to train and less prone to overfitting on smaller datasets. The trade-off is reduced representational capacity for very long sequences. For SCADA environments with limited training data, a GRU may outperform an LSTM despite its architectural simplicity.

A training paradigm where the LSTM receives a sequence of past SCADA commands as input and predicts the next expected command in the sequence. The model learns the normal grammar of industrial operations—for example, that a 'close breaker' command typically follows a 'sync check' command. When the actual next command deviates significantly from the prediction, the system flags it as anomalous. This approach captures both temporal order and inter-command dependencies that rule-based whitelisting misses.

A configuration where two independent LSTM layers process the input sequence in both forward and reverse directions simultaneously. The outputs are concatenated, giving the model access to both past and future context at each timestep. In offline SCADA forensic analysis, a bidirectional LSTM can examine an entire captured command sequence to identify anomalies that only become apparent when considering subsequent commands—such as a reconnaissance pattern that precedes a malicious write operation.