Humidity Correction for Low-Cost PM2.5 Sensors #

Humidity is the largest single error term in a low-cost PM2.5 network, larger than sensor noise and larger than the calibration residual — and unlike those, it is not random. It is a systematic over-reading that scales with how damp the air is, which means it correlates with weather, with time of day, and with exactly the conditions under which a network’s numbers get scrutinised. Correcting it is the highest-value normalization step available, and it must run before the linear cross-calibration described in cross-device sensor normalization techniques, because otherwise the regression absorbs an average humidity effect into a constant gain.

The Physics, Briefly #

An optical particle counter infers mass from scattered light. Ambient aerosol contains hygroscopic material — sulphates, nitrates, sea salt — that takes up water as relative humidity rises. The particle grows, its scattering cross-section grows faster than its dry mass, and the instrument reports a concentration inflated by water that a gravimetric reference dries away before weighing.

The kappa-Köhler framework parameterises this with a single hygroscopicity parameter κ. The growth factor for the reported mass is approximately:

C_dry = C_wet / (1 + κ · a_w / (1 − a_w)) where a_w = RH/100

κ near 0.2 suits continental urban aerosol; 0.4 to 0.6 suits marine or heavily sulphate-influenced air. The term a_w/(1−a_w) is what makes the correction explosive near saturation: at 50% humidity it is 1.0, at 90% it is 9.0, and at 98% it is 49. That behaviour is real — and it is also why the correction stops being trustworthy at the top of the range.

Production-Ready Implementation #

# python 3.11 · pandas==2.2.2 · numpy==1.26.4 · scikit-learn==1.4.2
from __future__ import annotations

from dataclasses import dataclass

import numpy as np
import pandas as pd

QC_GOOD, QC_SUSPECT, QC_HIGH_RH = 1, 2, 6


@dataclass(frozen=True)
class HumidityCorrection:
    """kappa-Koehler correction for hygroscopic growth in optical PM measurements.

    kappa: aerosol hygroscopicity. 0.2 continental urban, 0.4-0.6 marine.
    rh_max: above this the growth factor is too sensitive to trust — flag instead.
    """

    kappa: float = 0.22
    rh_max: float = 95.0
    rh_floor: float = 35.0        # below this, growth is negligible; leave the value alone

    def growth_factor(self, rh_pct: np.ndarray) -> np.ndarray:
        """Multiplicative inflation of reported mass at each humidity."""
        a_w = np.clip(rh_pct, 0.0, 99.5) / 100.0
        return 1.0 + self.kappa * a_w / (1.0 - a_w)

    def apply(self, df: pd.DataFrame, pm_col: str = "pm25",
              rh_col: str = "humidity_pct") -> pd.DataFrame:
        """Return the frame with a dry-mass estimate and a flag above rh_max."""
        out = df.copy()
        rh = out[rh_col].to_numpy(dtype=float)
        pm = out[pm_col].to_numpy(dtype=float)

        gf = self.growth_factor(rh)
        corrected = np.where(rh > self.rh_floor, pm / gf, pm)

        too_wet = rh > self.rh_max
        corrected = np.where(too_wet, np.nan, corrected)

        out[f"{pm_col}_dry"] = corrected
        out.loc[too_wet, "qc_flag"] = np.maximum(out.loc[too_wet, "qc_flag"], QC_HIGH_RH)
        out["growth_factor"] = gf
        return out

The rh_floor is not physics, it is engineering: below about 35% the correction is under 1.2% and applying it only adds the humidity sensor’s own error to a clean measurement.

Where co-location data exists, fit the relationship instead of assuming it:

from sklearn.linear_model import HuberRegressor


def fit_empirical_correction(
    df: pd.DataFrame, *, pm_col: str = "pm25", rh_col: str = "humidity_pct",
    ref_col: str = "reference_pm25",
) -> tuple[HuberRegressor, float]:
    """Regress the observed inflation ratio on the Koehler growth term.

    Fitting ratio = 1 + k * a_w/(1-a_w) recovers an effective kappa for THIS
    sensor and THIS aerosol, absorbing the optics and the local particle mix
    together. Returns the model and its residual standard deviation.
    """
    usable = (
        df[ref_col].notna() & (df[ref_col] > 2.0)      # ratios are unstable near zero
        & df[pm_col].notna() & df[rh_col].between(20, 95)
    )
    d = df.loc[usable]
    a_w = d[rh_col].to_numpy(dtype=float) / 100.0
    x = (a_w / (1.0 - a_w)).reshape(-1, 1)
    y = d[pm_col].to_numpy(dtype=float) / d[ref_col].to_numpy(dtype=float)

    model = HuberRegressor(fit_intercept=True).fit(x, y)
    residual_sd = float(np.std(y - model.predict(x)))
    return model, residual_sd

Excluding reference values below 2 µg/m³ matters more than it looks: at very low concentrations the ratio’s denominator approaches zero and a handful of points dominate the fit, producing an effective κ that is fitted to noise.

Parameter Tuning Guide #

The bottom row loses data — typically 2 to 6% of readings in a temperate maritime climate, more in fog-prone valleys. That loss is honest; publishing a value divided by a growth factor of eight is not.

Verification and Testing #

# python 3.11 · pandas==2.2.2 · numpy==1.26.4 · pytest==8.2.0
def test_correction_reduces_humidity_correlation_in_the_residuals(colocation):
    """The acceptance test: after correction, error must not depend on humidity."""
    corrected = HumidityCorrection(kappa=0.22).apply(colocation)
    ok = corrected["pm25_dry"].notna() & corrected["reference_pm25"].notna()

    before = np.corrcoef(colocation.loc[ok, "pm25"] - colocation.loc[ok, "reference_pm25"],
                         colocation.loc[ok, "humidity_pct"])[0, 1]
    after = np.corrcoef(corrected.loc[ok, "pm25_dry"] - corrected.loc[ok, "reference_pm25"],
                        corrected.loc[ok, "humidity_pct"])[0, 1]

    assert abs(after) < 0.5 * abs(before)


def test_growth_factor_matches_published_values():
    c = HumidityCorrection(kappa=0.22)
    assert abs(float(c.growth_factor(np.array([50.0]))[0]) - 1.22) < 0.01
    assert abs(float(c.growth_factor(np.array([90.0]))[0]) - 2.98) < 0.05


def test_very_humid_readings_are_flagged_not_corrected():
    df = pd.DataFrame({"pm25": [40.0], "humidity_pct": [97.0], "qc_flag": [QC_GOOD]})
    out = HumidityCorrection().apply(df)
    assert np.isnan(out.loc[0, "pm25_dry"])
    assert out.loc[0, "qc_flag"] == QC_HIGH_RH

The first test is the one that decides whether the correction is working at all. A correction that lowers the mean error while leaving the residuals correlated with humidity has simply moved the bias around; the goal is to remove the dependence, not to make one summary number smaller.

Gotchas #

Correcting after the linear calibration. The gain and offset fitted on uncorrected data have already absorbed an average humidity effect. Correct for humidity first, then fit the transfer function on the dry-mass estimate, as cross-calibration assumes.

Using ambient humidity instead of the humidity at the sensor inlet. Some units heat their inlet or sit inside a warm enclosure, where the local humidity is materially lower than outside. Use the sensor’s own humidity channel where it has one.

A single κ across a whole region. Coastal and inland sites in the same network genuinely have different aerosol. If your sites span an environment boundary, fit κ per site.

Applying the correction to a reference-grade instrument. Gravimetric and beta-attenuation monitors condition the sample before measuring, so their output is already dry mass. Correcting them introduces exactly the bias you are trying to remove.

FAQ #

Related #

• Cross-Device Sensor Normalization Techniques — the normalization stage this correction belongs to
• Cross-Calibrating PM2.5 Monitors with Linear Regression — the gain-and-offset calibration that runs after this correction
• Uncertainty Quantification and Data Quality Scoring — where the residual humidity sensitivity enters the uncertainty budget