Minimum Redundancy Maximum Relevance (mRMR) is a filter method for feature selection that ranks features using mutual information to balance two competing objectives: maximizing relevance to the target variable and minimizing redundancy among selected features. Unlike wrapper methods, mRMR operates independently of any downstream classifier, making it computationally efficient for high-dimensional biomarker datasets where the number of features far exceeds the number of samples.
Glossary
Minimum Redundancy Maximum Relevance (mRMR)
A filter-based feature selection algorithm that selects an optimal subset of features by maximizing their statistical dependency on the target variable while simultaneously minimizing the mutual information shared among the selected features.
FEATURE SELECTION
What is Minimum Redundancy Maximum Relevance (mRMR)?
A filter-based feature selection algorithm that identifies an optimal subset of features by maximizing their statistical dependency on the target variable while simultaneously minimizing the mutual information shared among the selected features themselves.
The algorithm employs a greedy forward search, iteratively adding the feature with the highest relevance-redundancy trade-off. Relevance is quantified as the mutual information between a candidate feature and the target, while redundancy is the average mutual information between the candidate and already-selected features. This framework is widely applied in genomic biomarker discovery to identify compact, non-collinear gene signatures that maintain strong predictive power for disease classification.
FILTER METHOD
Key Characteristics of mRMR
Minimum Redundancy Maximum Relevance (mRMR) is a powerful filter-based feature selection algorithm that balances two competing objectives: selecting features with the highest statistical dependency on the target variable while simultaneously penalizing features that are highly correlated with each other.
01
Mutual Information Maximization
mRMR uses mutual information to quantify the non-linear dependency between variables. The relevance term calculates the average mutual information between each candidate feature and the target class. This captures arbitrary statistical relationships, not just linear correlations, making it effective for complex biological datasets where gene interactions are often non-linear.
02
Redundancy Minimization Penalty
The redundancy term computes the average mutual information between a candidate feature and all previously selected features. By subtracting this penalty from the relevance score, mRMR explicitly avoids selecting features that are surrogates for each other. This ensures the final feature set is compact and non-redundant, reducing multicollinearity in downstream models.
03
Incremental Greedy Search
mRMR employs a forward greedy search strategy rather than evaluating all possible feature subsets. Starting with an empty set, it iteratively adds the feature that maximizes the relevance-redundancy trade-off. While not guaranteed to find the global optimum, this heuristic is computationally tractable for high-dimensional genomic datasets with tens of thousands of features.
04
MID vs MIQ Schemes
Two common operational schemes exist for combining relevance and redundancy:
- MID (Mutual Information Difference): Maximizes
relevance - redundancy - MIQ (Mutual Information Quotient): Maximizes
relevance / redundancy
MIQ tends to select features with higher individual relevance even if they carry some redundancy, while MID is more aggressive in penalizing correlated features.
05
Model-Agnostic Preprocessing
As a filter method, mRMR operates independently of any subsequent classifier or regression model. It ranks features based solely on the intrinsic properties of the data. This makes it extremely fast and resistant to overfitting compared to wrapper methods, though it may miss complex feature interactions that a specific model could exploit.
06
Continuous & Discrete Data Handling
mRMR handles both continuous and categorical variables by discretizing continuous features into bins before computing mutual information. Common approaches include equal-width or equal-frequency binning. The algorithm is also extensible to multi-class problems by using the joint mutual information between features and the full class vector.
mRMR EXPLAINED
Frequently Asked Questions
Clear, technically precise answers to the most common questions about the Minimum Redundancy Maximum Relevance algorithm for high-dimensional biomarker discovery.
Minimum Redundancy Maximum Relevance (mRMR) is a filter-based feature selection algorithm that selects an optimal subset of features by maximizing their statistical dependency with the target variable (relevance) while simultaneously minimizing the mutual information shared among the selected features themselves (redundancy).
It operates iteratively using a greedy search. Starting with an empty set, the algorithm evaluates all remaining features and selects the one that maximizes the objective function: max[I(f_i; c) - (1/|S|) * sum(I(f_i; f_j))], where I is mutual information, f_i is a candidate feature, c is the target class, and S is the set of already selected features. This ensures each new feature adds unique, non-overlapping information about the target.
- Relevance: Quantified by the mutual information between a feature and the target variable.
- Redundancy: Quantified by the average mutual information between a candidate feature and all previously selected features.
- Incremental Search: The process continues until a pre-specified number of features
kis reached, making it computationally efficient for high-dimensional genomic datasets.
FILTER METHOD COMPARISON
mRMR vs. Other Feature Selection Methods
A comparison of mRMR against other widely used filter, wrapper, and embedded feature selection techniques for high-dimensional biomarker data.
| Feature | mRMR | LASSO (L1) | Boruta | PCA |
|---|---|---|---|---|
Selection Paradigm | Filter (Mutual Information) | Embedded (L1 Penalty) | Wrapper (Random Forest) | Filter (Variance Maximization) |
Handles Feature Redundancy | ||||
Requires Target Variable | ||||
Output Interpretability | Original features retained | Original features retained | Original features retained | Linear combinations (latent) |
Computational Complexity | O(N^2) pairwise MI | O(N * P) coordinate descent | O(T * sqrt(P) * N) | O(min(N^3, P^3)) SVD |
Scalability (P > 10,000) | Moderate (pre-filtering advised) | High | Low (computationally intensive) | Moderate |
Hyperparameter Sensitivity | Low (k features to select) | High (regularization strength λ) | Moderate (shadow feature threshold) | Low (variance explained threshold) |
Captures Non-Linear Dependencies |
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.
Enterprise searchRAGPermissions
Read more
Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.
AI agentsWorkflow automationGovernance
Read more
Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
AI integrationDecision supportModel routing
Read more
About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
LinkedIn
Limited slotsGet a Free AI Consultation
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us