TransformerCPI

The drug compound is represented as a 2D molecular graph where atoms are nodes and bonds are edges. A Gated Graph Neural Network (GGNN) processes this graph through multiple propagation steps, using a gated recurrent unit (GRU) mechanism to update node hidden states. This allows the model to capture local chemical substructures and topological features. The final molecular representation is obtained via a readout function that aggregates all node embeddings into a single fixed-length feature vector, preserving the graph's structural integrity.

Instead of using a full transformer encoder-decoder, TransformerCPI employs a transformer decoder to process raw protein amino acid sequences. The decoder uses multi-head self-attention mechanisms to capture long-range dependencies between residues that may be far apart in the primary sequence but spatially proximal in the folded 3D structure. Positional encodings are added to the input embeddings to retain sequential order. The decoder's output is pooled to produce a fixed-dimensional protein representation, enabling the model to learn semantic features without requiring multiple sequence alignments or evolutionary profiles.

The core innovation lies in how the two modalities interact. The molecular graph embedding from the GGNN is fed as the initial input to the transformer decoder, which then attends to the protein sequence. This design allows the drug representation to act as a query while the protein sequence serves as the key and value in the cross-attention layers. This asymmetric interaction mechanism enables the model to learn which parts of the protein sequence are most relevant to the specific drug compound, effectively performing a form of soft binding pocket identification without explicit 3D structural data.

The final hidden states from the transformer decoder are passed through a fully connected feed-forward network to produce the prediction. For binary classification (interaction vs. non-interaction), a sigmoid activation outputs a probability score. For regression tasks like binding affinity prediction, a linear layer outputs a continuous value. The entire architecture is trained end-to-end with binary cross-entropy loss or mean squared error loss, allowing gradients to flow from the prediction head back through both the protein transformer and the molecular graph encoder simultaneously.

A critical architectural consideration is the negative sampling methodology. Since experimentally validated non-interacting pairs are scarce, TransformerCPI employs a strategy where unlabeled drug-target pairs are randomly sampled as negatives during training. The model uses a balanced sampling ratio (typically 1:1 positive to negative) to prevent class imbalance. This approach implicitly assumes that the vast majority of random drug-target pairings are non-interacting, a reasonable prior given the sparsity of the full interaction matrix. The sampling is performed dynamically per epoch to expose the model to diverse negative examples.

Drug compounds are tokenized as molecular graphs with atom features (atom type, degree, formal charge, chirality, hybridization, aromaticity) and bond features (bond type, conjugation, ring membership). Protein sequences are tokenized at the amino acid level using a learned embedding layer with a vocabulary of 20 standard residues plus special tokens. The model does not rely on pre-computed molecular fingerprints (like ECFP) or position-specific scoring matrices (PSSM), making it fully differentiable and capable of learning task-specific representations directly from raw sequence and graph data.

The specific physical binding event between a drug molecule and a cellular macromolecular target, such as a protein or nucleic acid, that initiates a pharmacological effect. TransformerCPI is explicitly designed to predict these interactions by modeling the semantic relationships between a protein's amino acid sequence and a compound's molecular graph.

• Key Concept: The input is a pair (protein, compound); the output is a binary interaction label or a continuous binding affinity score.
• TransformerCPI's Role: It frames DTI prediction as a sequence-to-graph classification problem, using a transformer to decode protein context for the molecular graph.

The quantitative strength of the non-covalent interaction between a biomolecule, typically a protein, and its ligand, usually expressed via thermodynamic dissociation constants (Kd) or inhibition constants (Ki). TransformerCPI's gated convolutional architecture is optimized to regress these continuous values.

• Measurement: Often represented as pKd or pKi (negative log of the constant) for regression tasks.
• TransformerCPI Advantage: The model captures long-range intermolecular dependencies that simpler 1D convolutional networks miss, leading to more accurate affinity predictions.

A foundational deep learning architecture that uses two separate convolutional neural networks (CNNs) to learn feature representations from raw protein sequences and drug SMILES strings for predicting binding affinity. It serves as a critical baseline against which TransformerCPI is compared.

• Limitation: DeepDTA processes proteins and drugs with independent 1D CNNs, failing to model cross-modal interactions between specific amino acid residues and molecular substructures.
• TransformerCPI's Innovation: Replaces the protein CNN with a transformer decoder that attends directly to the molecular graph's atom representations, enabling pairwise interaction learning.

A deep learning method that represents drug molecules as 2D molecular graphs processed by a graph neural network (GNN), combined with a protein sequence CNN, to improve binding affinity prediction accuracy. It acknowledges that graphs are a more natural representation of molecular topology than linear SMILES strings.

• Advancement: Uses graph isomorphism networks (GINs) or GCNs to learn atom-level embeddings, capturing bond connectivity.
• TransformerCPI's Extension: While GraphDTA uses a GNN for the drug, TransformerCPI uses a gated graph convolutional network and pairs it with a transformer to model how the protein sequence contextually gates the molecular graph's features.

The critical process in machine learning for DTI prediction of selecting a representative set of non-interacting drug-target pairs to train a classifier, preventing a model from becoming biased by a lack of negative data. TransformerCPI's performance is heavily dependent on a robust negative sampling strategy.

• Common Strategies: Random pairing of drugs and targets not known to interact, or using PU-learning (Positive-Unlabeled) frameworks.
• Impact on TransformerCPI: Poor negative sampling can lead to inflated accuracy metrics. The model must be trained on negatives that are physicochemically plausible but biologically inactive to learn a meaningful decision boundary.

A machine learning approach that simultaneously uses descriptors from both the ligand chemical space and the target protein sequence space to predict bioactivity across a large interaction matrix. TransformerCPI is a deep learning instantiation of this concept.

• Core Idea: A single model learns a joint embedding space where the similarity between a drug and a target correlates with their interaction likelihood.
• TransformerCPI's Alignment: The transformer's cross-attention mechanism explicitly creates a joint representation of the protein sequence and the molecular graph, making it a powerful proteochemometric engine.