Translation Quality Estimation (QE)

The defining characteristic of QE is its ability to assess translation quality without access to a human reference translation. Unlike metrics like BLEU or COMET, which require a gold-standard translation for comparison, QE models rely solely on the source text and the machine-translated target text. This is achieved through supervised learning on datasets where human annotators have directly labeled translations with quality scores, error spans, or post-editing effort indicators.

Granular QE systems output a quality tag for each token in the machine-translated text, typically using labels such as:

• OK: The token is correctly translated and requires no editing.
• BAD: The token is incorrect and must be deleted or replaced. These fine-grained predictions power downstream features like automatic word highlighting in computer-assisted translation (CAT) tools, guiding human post-editors directly to problematic spans.

At the sentence level, QE models predict the Human Translation Error Rate (HTER), which measures the amount of editing required to fix a machine translation. HTER is calculated as the minimum number of insertions, deletions, substitutions, and shifts needed to correct the output, divided by the reference length. A low predicted HTER indicates a high-quality translation that may require no human intervention, enabling dynamic workflow routing.

Advanced QE architectures incorporate document-level context to resolve ambiguities that are invisible at the sentence level. By processing surrounding sentences or the entire document, these models can detect discourse-level errors such as:

• Incorrect pronoun gender due to cross-sentence antecedents.
• Lexical inconsistencies where the same term is translated differently across paragraphs.
• Violations of formal/informal register maintained throughout a document.

A well-calibrated QE system outputs a confidence score that accurately reflects the empirical likelihood of correctness. Calibration ensures that when a model assigns a 90% confidence score to a set of translations, approximately 90% of them are indeed correct. This is critical for risk-based decisioning, such as setting thresholds for automated publishing versus mandatory human review, and is often measured using Expected Calibration Error (ECE).

Beyond binary OK/BAD labels, sophisticated QE systems classify errors into linguistically motivated categories. Common error types include:

• Addition: Extra words not present in the source.
• Omission: Source content missing from the translation.
• Mistranslation: Incorrect lexical or phrasal rendering.
• Untranslated: Source language text left in the output. This typology provides actionable diagnostics for improving the underlying NMT engine.

A machine learning task focused on automatically correcting errors in raw machine translation output. APE and QE form a natural pipeline: QE first identifies which segments or words are likely erroneous, and APE then applies targeted corrections. This tight coupling enables selective post-editing where only low-confidence spans are revised, dramatically reducing computational overhead compared to re-translating entire documents.

A broader machine learning concept that QE extends to the translation domain. Confidence estimation provides a calibrated probability score reflecting a model's certainty in its output. In QE, this manifests at multiple granularities:

• Word-level: Highlighting specific tokens likely to be incorrect
• Sentence-level: Overall quality score (e.g., 0-100)
• Document-level: Aggregate quality assessment for routing decisions Well-calibrated confidence scores are essential for risk-based routing in production translation pipelines.

A database storing previously translated segments in source-target language pairs. QE enhances TM workflows by scoring the quality of fuzzy matches retrieved from the TM. When a TM suggests a 90% fuzzy match, a QE model can independently verify whether that match is actually usable or requires significant rework. This prevents the common problem of garbage-in, garbage-out where poor TM suggestions degrade translator productivity.

A workflow paradigm where machine translation output is reviewed and corrected by human translators. QE serves as the intelligent triage layer in this loop:

• Routes high-confidence segments directly to publication
• Flags medium-confidence segments for light post-editing
• Sends low-confidence segments for full human re-translation This confidence-based routing optimizes human effort allocation, reducing translation costs by 30-50% while maintaining quality thresholds.