I/Q Mismatch Modeling

The core mathematical framework represents the impaired output as a 2×2 widely-linear matrix that maps the ideal complex baseband signal to the corrupted RF signal. This matrix captures both the direct signal path and the conjugate image path, which is the fundamental signature of I/Q imbalance. The model is expressed as:

• Direct component: The desired signal scaled by a complex coefficient
• Image component: The conjugate of the desired signal scaled by a mismatch coefficient
• Matrix form: [y_I; y_Q] = [[α_I, β_I]; [α_Q, β_Q]] × [x_I; x_Q]

The widely-linear structure is essential because standard linear filtering cannot compensate for the conjugate term that generates the image interference.

The simplest and most common model assumes static, narrowband imbalance where gain and phase errors remain constant across the entire signal bandwidth. This model uses:

• Gain imbalance parameter (ε): Ratio of I-channel gain to Q-channel gain, typically expressed in dB or as a fractional deviation from unity
• Phase imbalance parameter (φ): Deviation from the ideal 90° phase offset between I and Q local oscillator signals, measured in degrees
• Single complex coefficient: A single image rejection coefficient fully characterizes the impairment

This model is valid when the signal bandwidth is small relative to the modulator's analog bandwidth and when anti-aliasing filters are well-matched.

For wideband signals such as 5G NR and WiFi 6, the gain and phase errors vary across frequency, requiring a more sophisticated model. This frequency-dependent representation includes:

• Mismatched filter responses: Different transfer functions H_I(f) and H_Q(f) for the I and Q paths
• I/Q skew (τ): Relative timing delay between I and Q sampling clocks, causing linear phase distortion
• Cross-talk coefficients: Frequency-selective coupling between I and Q paths
• Complex FIR filter: A finite impulse response filter structure replaces the scalar mismatch coefficient

The model captures gain ripple and phase ripple across the band, which becomes critical when linearization bandwidths exceed 100 MHz in modern transmitters.

The Image Rejection Ratio (IRR) is the primary performance metric derived from the mismatch model, quantifying the power ratio between the desired signal and the unwanted image sideband:

• Formula: IRR = 10 × log₁₀(|α|² / |β|²) dB, where α is the direct path coefficient and β is the image path coefficient
• Gain-only imbalance: IRR ≈ 10 × log₁₀((2+ε)² / ε²) for small phase errors
• Phase-only imbalance: IRR ≈ 10 × log₁₀(4 / φ²) for small gain errors
• Combined effect: Both gain and phase errors degrade IRR, with phase imbalance typically dominating in well-designed modulators

A practical modulator without compensation typically achieves 25-35 dB IRR, while modern DPD systems target 50-60 dB IRR after correction.

Beyond gain and phase mismatch, the complete model incorporates DC offset components that manifest as local oscillator leakage at the carrier frequency:

• I-channel DC offset (c_I): Constant voltage added to the in-phase baseband signal
• Q-channel DC offset (c_Q): Constant voltage added to the quadrature baseband signal
• Composite LO leakage: The vector sum of c_I and c_Q modulated by the LO, producing a spurious tone at f_c
• Extended matrix model: [y_I; y_Q] = M × [x_I; x_Q] + [c_I; c_Q]

LO leakage is particularly problematic in direct conversion transmitters because it falls directly in-band and cannot be filtered, requiring explicit digital offset compensation.

In real transmitter chains, I/Q mismatch interacts with power amplifier nonlinearity in complex ways that cannot be modeled as independent cascaded impairments:

• AM-AM/AM-PM coupling: Gain and phase imbalance alter the signal envelope statistics, changing how the PA nonlinearity is excited
• Image regrowth: PA nonlinearity can regenerate image components even after I/Q predistortion
• Joint modeling requirement: Accurate wideband linearization requires a unified model that captures both I/Q mismatch and PA memory effects simultaneously
• Volterra-I/Q models: Extended Volterra series that include conjugate terms to represent the combined impairment

This interaction drives the need for joint DPD and I/Q compensation architectures rather than sequential correction stages.

The mathematical framework that models an impaired quadrature modulator as a system that processes both the signal and its complex conjugate. This is the core of I/Q mismatch modeling, expressed as y = K₁x + K₂x*.

• Matrix form: The 2×2 mismatch matrix maps [I_in, Q_in]ᵀ to [I_out, Q_out]ᵀ using real-valued coefficients
• Complex form: y = μx + νx*, where μ represents the direct path and ν represents the image-producing path
• Coefficient extraction: μ = cos(φ/2) + jα sin(φ/2), ν = α cos(φ/2) + j sin(φ/2)
• Compensation principle: Applying the inverse matrix, x_corrected = (μy - νy) / (|μ|² - |ν|²), restores the original signal