Glossary

Rocchio Algorithm

A classic relevance feedback algorithm that reformulates a query vector by adding the centroid of relevant document vectors and subtracting the centroid of non-relevant ones to improve information retrieval precision.

Get in touch Learn more

Developer working on RAG retrieval system, document chunks visible on screen, technical workspace with code editor.

RELEVANCE FEEDBACK

What is Rocchio Algorithm?

The Rocchio algorithm is a classic vector-space method for query reformulation that iteratively improves search results by incorporating explicit user feedback on document relevance.

The Rocchio algorithm is a relevance feedback technique that modifies an original query vector by adding the centroid of relevant document vectors and subtracting the centroid of non-relevant document vectors. This linear adjustment moves the query point in the vector space toward a region of high relevance density, improving retrieval precision in subsequent iterations.

Named after Joseph Rocchio, the algorithm applies tunable weights (alpha, beta, gamma) to control the influence of the original query, relevant documents, and non-relevant documents respectively. While foundational in information retrieval, its reliance on explicit user judgments limits scalability, leading modern systems to favor implicit signals or pseudo-relevance feedback for automatic query expansion.

Relevance Feedback Mechanics

Key Characteristics of the Rocchio Algorithm

The Rocchio algorithm is a classic vector space model technique for query reformulation. It iteratively refines a query vector by moving it toward the centroid of relevant documents and away from the centroid of non-relevant documents.

The Core Formula

The algorithm is defined by the formula: Q_new = α * Q_orig + β * (Σ Relevant / |Rel|) - γ * (Σ Non-Relevant / |NonRel|). The parameters α, β, and γ control the weight of the original query, the positive feedback from relevant documents, and the negative feedback from non-relevant documents. Tuning these weights is critical to balancing precision and recall.

Vector Space Operation

Rocchio operates in a vector space model where documents and queries are represented as TF-IDF weighted vectors. The algorithm performs vector addition and subtraction to shift the query point in high-dimensional space. The goal is to move the query vector into a region of the space with a higher density of relevant documents and away from clusters of non-relevant ones.

Relevance Feedback Dependency

This is a supervised query expansion method that requires explicit relevance judgments. A user or system must identify a set of documents as relevant and a set as non-relevant from an initial retrieval. The quality of the reformulated query is directly proportional to the quality and quantity of this feedback, making it highly effective in iterative search scenarios.

Pseudo-Relevance Feedback (PRF)

To automate the process without user input, Rocchio is often used in Pseudo-Relevance Feedback. The system blindly assumes the top-k initially retrieved documents are relevant and the bottom-k are non-relevant. While efficient, this creates a risk of query drift if the initial top documents are not actually relevant, causing the query to veer off-topic.

Computational Efficiency

The algorithm is computationally lightweight. It only requires simple vector arithmetic on sparse representations, making it much faster than neural re-ranking methods. This efficiency made it a standard baseline for query expansion in early search engines and still makes it viable for real-time systems with strict latency budgets.

Limitations in Modern Search

Rocchio struggles with multimodal relevance—if relevant documents form distinct clusters, the centroid may fall in empty space. It also fails to capture semantic similarity beyond exact term overlap. Modern systems often replace it with contextualized embedding expansion or generative models that can add novel synonyms not present in the feedback documents.

ROCCHIO ALGORITHM

Frequently Asked Questions

Clear, technical answers to the most common questions about the Rocchio algorithm, its mechanism, parameters, and role in modern relevance feedback systems.

The Rocchio algorithm is a classic relevance feedback technique in information retrieval that reformulates a user's query vector by moving it toward the centroid of relevant documents and away from the centroid of non-relevant documents. It operates in the vector space model, where both queries and documents are represented as weighted term vectors. The core formula is: Q_new = α * Q_original + β * (1/|Dr| * Σ Dj in Dr) - γ * (1/|Dnr| * Σ Dk in Dnr), where Dr is the set of relevant documents, Dnr is the set of non-relevant documents, and α, β, and γ are tunable weights controlling the influence of the original query, positive feedback, and negative feedback respectively. The algorithm effectively adds terms from relevant documents to the query and subtracts terms from non-relevant ones, improving both recall and precision in subsequent retrieval iterations.

RELEVANCE FEEDBACK COMPARISON

Rocchio Algorithm vs. Pseudo-Relevance Feedback

A direct comparison of the explicit, user-driven Rocchio Algorithm and the automatic, assumption-based Pseudo-Relevance Feedback method for query refinement.

Feature	Rocchio Algorithm	Pseudo-Relevance Feedback
Feedback Source	Explicit user judgments	Top-k initial retrieval results
User Involvement
Risk of Query Drift	Low (controlled by user)	High (unchecked expansion)
Latency Overhead	User interaction time	Second-pass retrieval time
Parameter Sensitivity	α, β, γ weights	k (top docs), m (top terms)
Relevance Certainty	High (human-verified)	Low (assumed relevant)
Typical Use Case	Patent search, legal discovery	Web search, ad-hoc retrieval

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

QUERY REFINEMENT ECOSYSTEM

Related Terms

The Rocchio algorithm is a foundational relevance feedback method. Explore the related techniques that complement, extend, or contrast with this classic vector-space approach to query reformulation.

Relevance Feedback

The iterative search paradigm within which the Rocchio Algorithm operates. Users explicitly mark initial results as relevant or non-relevant, and the system uses these judgments to reformulate the query.

Explicit Judgments: Requires direct user input on result quality.
Vector Modification: The core mechanism involves moving the query vector toward relevant document centroids.
Contrast: Unlike Pseudo-Relevance Feedback, this relies on human assessment, not automated assumptions.

1971

Year Introduced

Pseudo-Relevance Feedback

An automatic alternative to the Rocchio Algorithm that assumes the top-k initially retrieved documents are relevant, bypassing the need for explicit user judgments.

Blind Feedback: Automatically treats the first retrieval pass as ground truth.
Efficiency: Removes the human-in-the-loop requirement, enabling real-time query drift.
Risk: Susceptible to query drift if the initial top-k results contain non-relevant documents, poisoning the reformulated query.

EXPLORE

Query Expansion

The broader category of techniques that enrich a user's query with additional terms. The Rocchio Algorithm achieves this by extracting discriminative terms from the centroid of relevant documents.

Synonym Expansion: Adds words with identical meanings.
Hypernym/Hyponym Expansion: Broadens or narrows the query using hierarchical relationships.
Rocchio's Role: A classic vector-based method for selecting expansion terms from relevant document clusters.

BM25 Algorithm

A probabilistic retrieval function often used as the initial scoring mechanism before applying the Rocchio Algorithm. BM25 provides the baseline relevance scores for the first-pass retrieval.

Sparse Retrieval: Relies on exact term frequency and inverse document frequency.
Rocchio Integration: The Rocchio formula modifies the query vector, which is then re-executed against the BM25 index.
Contrast: BM25 is a static scoring function, while Rocchio dynamically adapts the query representation.

Query Drift

A critical failure mode of the Rocchio Algorithm where the reformulated query vector shifts away from the original information need, often caused by poor initial relevance judgments.

Cause: Adding terms from non-relevant documents or over-weighting a specific aspect of relevant documents.
Mitigation: Applying query anchoring by retaining a weighted component of the original query vector.
Parameter Tuning: The alpha, beta, and gamma weights in the Rocchio formula directly control the resistance to drift.

Contextualized Embedding Expansion

A modern neural alternative to the Rocchio Algorithm that uses deep language models like BERT to generate expansion terms based on the query's specific context, rather than statistical term distributions.

Semantic Understanding: Captures polysemy and context, which the bag-of-words Rocchio model misses.
Vector Comparison: Operates in dense embedding spaces rather than sparse TF-IDF vectors.
Evolution: Represents the shift from explicit vector arithmetic to learned neural representations for query reformulation.

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.