Spatial Domain Detection

The core mechanism constructs a spatial neighborhood graph where nodes represent capture locations (spots or cells) and edges connect k-nearest spatial neighbors. Clustering algorithms like Leiden or Louvain then partition this graph to maximize intra-domain expression similarity while respecting spatial contiguity. This approach enforces that domains are not just transcriptionally similar but also physically connected, preventing fragmented or biologically implausible regions.

A probabilistic framework that assumes each spatial location has an unobserved hidden state (the domain label) and that the observed gene expression depends on this state. Crucially, the model incorporates a Markov assumption: the domain label at one location depends only on its immediate neighbors. Methods like SpaGCN and BayesSpace use this to smooth noisy expression data, enhancing domain boundaries that align with histological structures while suppressing technical artifacts.

Advanced detectors fuse heterogeneous data streams to improve domain accuracy:

• Gene expression: Normalized count matrices from spatial transcriptomics
• Histology images: Pixel-level features extracted via pre-trained convolutional neural networks
• Spatial coordinates: Physical x,y positions encoding tissue architecture Joint latent space models learn shared representations where domains are defined by both molecular and morphological coherence, critical for identifying boundaries invisible to expression data alone.

Methods like STAGATE and SpaceFlow employ graph attention autoencoders that learn low-dimensional embeddings capturing both gene expression and spatial context. The network aggregates information from neighboring nodes through message-passing layers, producing spatially-aware latent representations. These embeddings are then clustered to identify domains. The deep learning advantage lies in capturing non-linear spatial dependencies and complex tissue architectures that linear methods miss.

Domain quality is quantitatively assessed using spatial statistics:

• Moran's I: Measures whether domain assignments exhibit significant spatial clustering versus random distribution
• Geary's C: Evaluates local spatial autocorrelation, sensitive to sharp boundaries
• Adjusted Rand Index: Compares detected domains against expert histological annotations when ground truth exists High Moran's I values confirm that domains are not merely transcriptional clusters but genuinely spatially coherent structures.

Domain detection is fundamentally scale-dependent. The number of domains identified is controlled by clustering resolution parameters or the number of hidden states in probabilistic models. Fine resolutions reveal subtle sub-domains like cortical layers, while coarse resolutions identify major anatomical divisions. Multi-scale approaches iteratively detect domains at varying granularities, building hierarchical tissue atlases that capture both broad regions and fine-grained functional units.

Constructs a spatial neighborhood graph where nodes represent spots or cells and edges connect spatial neighbors. Algorithms like Louvain or Leiden community detection then partition this graph to identify domains with similar transcriptomic profiles.

• Leverages both gene expression similarity and physical adjacency
• Scales efficiently to datasets with hundreds of thousands of spatial locations
• Commonly used in tools like Seurat, Scanpy, and Squidpy
• Graph construction parameters (k-nearest neighbors, distance thresholds) critically influence domain granularity

Probabilistic models that assume observed gene expression at each location depends on an unobserved hidden state representing a spatial domain. Transitions between states follow a Markov random field, enforcing spatial smoothness.

• Models spatial dependencies explicitly through transition probability matrices
• BayesSpace uses a t-distributed error model with spatial priors for subspot resolution enhancement
• Naturally handles uncertainty quantification for domain assignments
• Particularly effective when tissue architecture follows smooth anatomical gradients

Neural network architectures that learn domain representations by integrating spatial context directly into the model. Spatial graph neural networks and convolutional autoencoders capture complex, non-linear relationships between gene expression and tissue architecture.

• SpaGCN combines graph convolution with attention mechanisms to identify domains
• STAGATE uses graph attention autoencoders for spatially aware dimensionality reduction
• Can incorporate histological image features alongside transcriptomic data
• Requires careful regularization to avoid overfitting to technical noise

Decomposes the spatial expression matrix into metagenes or factors that capture co-expression signatures associated with specific tissue domains. Each spatial location is represented as a weighted combination of these factors.

• Spatial NMF variants incorporate spatial smoothness penalties on factor loadings
• Produces interpretable factors that often correspond to known cell types or tissue structures
• Giotto implements spatial-aware NMF with HMRF priors
• Enables soft domain assignments where spots can belong partially to multiple regions

Bayesian non-parametric models that place Gaussian process priors over spatial coordinates to model smoothly varying gene expression patterns. Domains emerge from shared latent spatial functions.

• SpatialDE uses Gaussian process regression to identify spatially variable genes as a precursor to domain detection
• GPcounts models discrete count data with Gaussian process likelihoods
• Provides rigorous uncertainty quantification through posterior distributions
• Computationally intensive for large-scale datasets but highly statistically principled

Algorithms that jointly analyze spatial transcriptomics with histology images, immunofluorescence, or spatial proteomics to define domains. These methods learn shared latent representations across modalities.

• MUSE integrates morphology features from H&E images with gene expression
• Spatial-MultiOmics frameworks align multiple spatial assays to a common coordinate system
• Deep learning fusion layers combine image embeddings with transcriptomic features
• Improves domain boundary detection where transcriptomic signals are subtle but histological transitions are sharp

A fundamental statistical measure quantifying the degree to which a gene's expression at one location depends on the values at nearby locations. Positive spatial autocorrelation indicates that similar expression levels cluster together, forming the statistical basis for domain detection.

• Moran's I: A global statistic ranging from -1 (dispersed) to +1 (clustered), used to test if a pattern is random.
• Geary's C: A related measure that is more sensitive to local differences between neighbors.
• Algorithms like SpatialDE and SPARK use Gaussian process regression to identify genes with significant spatial autocorrelation.

A probabilistic generative model that assumes the tissue is composed of a finite number of unobserved spatial domains (hidden states). Each domain has a characteristic gene expression profile, and the state at a location depends on the states of its neighbors.

• The transition probabilities between neighboring states enforce spatial smoothness in the domain assignments.
• The emission probabilities model the observed gene expression counts given the hidden domain.
• Tools like BayesSpace use a spatial HMM to enhance the resolution of spot-based spatial transcriptomics data and identify coherent tissue regions.

A data structure that represents the tissue as a network, where each node is a spatial location (cell or spot) and edges connect neighboring locations. This graph is the computational foundation for most spatial domain detection algorithms.

• Constructed using k-nearest neighbors (k-NN) in Euclidean space or a fixed distance radius.
• Edges can be weighted by distance decay or left unweighted for adjacency-based methods.
• Graph-based clustering algorithms like Leiden or Louvain are then applied to this graph to partition the tissue into domains with coherent expression profiles.

A complementary computational approach that, instead of identifying discrete domains, estimates the proportion of different cell types within each spatial location. This is critical when a single spot captures mRNA from multiple cells.

• Reference-based methods (e.g., RCTD, SPOTlight) use a single-cell RNA-seq atlas as a signature matrix to deconvolve spot-level mixtures.
• Reference-free methods (e.g., STdeconvolve) infer latent cell-type proportions directly from the spatial data without an external atlas.
• The resulting cell-type proportion maps can be used as features for downstream domain detection or to annotate domains with their cellular composition.

The process of partitioning a digital tissue image (e.g., H&E stain) into distinct anatomical or functional regions based on pixel-level visual features. This provides an orthogonal, histology-driven definition of spatial domains.

• Deep learning models like U-Net and Mask R-CNN are trained to classify every pixel into tissue compartments (e.g., tumor, stroma, necrosis).
• The resulting segmentation masks can be integrated with gene expression data to define histology-guided domains.
• This approach anchors computational domains to visually interpretable tissue architecture, which is critical for pathologist validation.

A computational method that orders cells or spots based on their spatial coordinates and gene expression profiles to reconstruct dynamic biological processes directly in tissue context, such as differentiation or tumor invasion.

• Unlike pseudotime analysis in dissociated single-cell data, spatial trajectory inference constrains the ordering by physical proximity.
• Tools like SpaceFlow and stLearn use graph-based pseudo-time algorithms on the spatial neighborhood graph to identify transitional gradients.
• This reveals continuous gene expression gradients that may not be captured by discrete domain clustering, identifying boundary regions and migratory paths.