t-SNE

t-SNE converts high-dimensional Euclidean distances into conditional probabilities representing similarities. For each pair of datapoints, it computes the probability that one would pick the other as its neighbor under a Gaussian distribution centered on itself. This creates an asymmetric probability matrix in the high-dimensional space, while a Student t-distribution with one degree of freedom is used in the low-dimensional space to alleviate the 'crowding problem' and allow moderate distances to be faithfully represented.

The perplexity parameter is a smooth measure of the effective number of neighbors and is arguably the most critical tuning knob in t-SNE. It is related to the variance of the Gaussian kernel used in the high-dimensional space:

• Low perplexity (5-10): Focuses on very local structure, revealing fine clusters but potentially fragmenting larger groups.
• High perplexity (50-100): Captures broader global relationships but may blur fine details. Typical values range from 5 to 50, and the algorithm is generally robust to selections within this range for datasets of standard size.

The objective of t-SNE is to minimize the Kullback-Leibler (KL) divergence between the joint probability distribution in the high-dimensional space (P) and the joint probability distribution in the low-dimensional map (Q). This asymmetric cost function heavily penalizes placing dissimilar datapoints close together (false positives) while being more lenient about placing similar datapoints far apart. This property is what gives t-SNE its exceptional ability to preserve local clusters at the expense of potentially distorting global geometry.

The low-dimensional map is optimized iteratively using a variant of gradient descent with momentum. The gradient has a physical interpretation as a set of springs between points: attractive forces pull similar points together, while repulsive forces push all points apart. The optimization typically employs:

• Early exaggeration: Temporarily multiplying the attractive forces to create widely separated clusters initially.
• Learning rate adaptation: An adaptive learning rate scheme to stabilize convergence. The final embedding is non-deterministic; multiple runs with different random seeds are standard practice.

In Radio Frequency Machine Learning, t-SNE is a critical tool for visualizing learned representations from neural networks processing raw IQ data. Engineers use it to:

• Validate learned features: Confirm that embeddings from a neural receiver or classifier form distinct, separable clusters by modulation type or device identity.
• Detect anomalies: Identify outlier signal samples that do not cluster with known classes, indicating potential jamming or novel emitters.
• Debug training: Visually assess whether a model has collapsed representations or failed to discriminate between specific signal classes.

t-SNE has well-documented limitations that must be respected in technical analysis:

• No global structure preservation: Cluster sizes and distances between clusters are meaningless; only proximity within clusters is interpretable.
• Non-parametric nature: t-SNE does not learn a mapping function, so new points cannot be embedded without re-running the entire algorithm.
• Scalability: The standard Barnes-Hut implementation reduces complexity to O(N log N), but very large datasets still require subsampling or alternative techniques like UMAP.
• Deterministic randomness: Always fix the random seed for reproducibility and run multiple initializations to avoid interpreting stochastic artifacts as structure.

A hyperparameter that balances attention between local and global aspects of the data. It is a smooth measure of the effective number of neighbors.

• Low perplexity (5-10): Focuses on very local clusters; can fragment continuous manifolds.
• High perplexity (50-100): Considers broader structure; may merge distinct clusters.
• Heuristic: Typical values range from 5 to 50, often set roughly to the expected cluster size.

The precursor to t-SNE that converts high-dimensional Euclidean distances into conditional probabilities representing similarities.

• Crowding Problem: SNE suffers from a cost function that makes it difficult to separate moderately dissimilar points in the low-dimensional map.
• Asymmetric Cost: Uses a Kullback-Leibler divergence that is not symmetric, leading to a different optimization landscape than t-SNE.

Uniform Manifold Approximation and Projection is a competing algorithm that often preserves more of the global structure and runs significantly faster than t-SNE.

• Topological Foundation: Based on Riemannian geometry and algebraic topology.
• Key Difference: Better at preserving the continuity of large-scale data structures, not just local neighborhoods.
• Scalability: Handles millions of points more efficiently than standard t-SNE implementations.

A linear dimensionality reduction technique often used as a preprocessing step for t-SNE to reduce noise and accelerate computation.

• Linear Constraint: Only captures global variance via orthogonal transformation; cannot unfold non-linear manifolds.
• Initialization: PCA initialization for t-SNE embedding coordinates ensures deterministic and often more stable results.
• Variance Ratio: Provides a quantitative measure of how much information is retained in the reduced space.

The asymmetric statistical distance minimized by t-SNE to make the low-dimensional similarity distribution match the high-dimensional one.

• Gradient Interpretation: Creates an attractive force to pull similar points together and a repulsive force to push dissimilar points apart.
• Heavy-Tailed Student-t: The t-distribution in the low-dimensional space gives t-SNE its name and alleviates the crowding problem by allowing moderate distances to be modeled more faithfully.

An approximation algorithm that reduces the computational complexity of t-SNE from O(N²) to O(N log N).

• Mechanism: Uses a quadtree or octree to group distant points and treat them as a single effective repulsive force.
• Trade-off: Introduces a small approximation error (controlled by the theta parameter) in exchange for massive speed gains on datasets with thousands of points or more.