Decoding Cayenne LPP Payloads into a Pandas DataFrame

Cayenne Low Power Payload (LPP) encodes several sensor readings into one compact LoRaWAN uplink as a flat run of (channel, type, data...) groups, and decoding it into a typed pandas DataFrame means walking those bytes, applying the correct scale and sign per type, and emitting one tidy row per channel. Because the type byte fixes both the data width and the resolution, the decoder needs no external schema — it is self-describing — but it does need exact integer handling: a missed sign bit or a wrong scale divisor produces a physically plausible, silently wrong number. This page gives a self-contained struct-based decoder that turns a raw frame into a DataFrame, and it slots directly into the broader LoRaWAN payload parsing workflow.

How Cayenne LPP Encodes Channels

Each measurement in an LPP frame is prefixed by two header bytes: a channel byte that lets one device multiplex many sensors (channel 1 = the onboard thermistor, channel 2 = an external probe), and a type byte that identifies the quantity and thereby the number of data bytes that follow and how to scale them. A frame carrying a temperature and a humidity is just 01 67 <2 bytes> 02 68 <1 byte> — five channels of readings pack into well under the airtime budget of a single spreading-factor-7 uplink.

The scaling is where correctness lives. Integer fields are stored at fixed resolution to avoid transmitting floats: temperature in tenths of a degree, humidity in half-percent steps, GPS coordinates in ten-thousandths of a degree. Multi-byte fields are big-endian, and several types — temperature, analog input, GPS — are two’s-complement signed so they can represent sub-zero readings and southern/western coordinates. The decoder must honour both the width and the sign for every type, which is why a lookup table keyed by type id is cleaner than a tower of if branches.

The self-describing property is what makes LPP attractive for heterogeneous environmental fleets. A soil station, an air-quality node, and a weather mast can share one decoder even though they report entirely different quantities, because each measurement announces its own type and the decoder simply reads whatever the frame contains. That flexibility has a cost: the frame carries no length prefix and no checksum, so the decoder cannot know in advance how many channels to expect. It must advance a cursor two header bytes at a time, look up the data width from the type, and stop cleanly when the buffer is exhausted. Any assumption about total frame length — “our devices always send temperature then humidity” — becomes a latent bug the first time a device omits a channel because a probe was unplugged or a GPS module never achieved lock.

Choosing pandas as the output container is deliberate rather than incidental. A DataFrame gives every channel an explicit dtype, so a downstream spatial join never has to guess whether value is an object column of mixed ints and floats. It also makes multi-uplink batching a one-line pd.concat, and it lets validation run as vectorized boolean masks — checking that all temperatures fall inside a physical range across a thousand uplinks is a single comparison rather than a Python loop.

Production-Ready Implementation

The function below decodes a complete Cayenne LPP frame into a pandas DataFrame with one row per channel. It handles temperature, humidity, analog input, and GPS with their correct scaling, uses only the standard library plus pandas, and validates the frame bounds so a truncated uplink raises rather than reading past the buffer.

# python 3.11 · pandas==2.2.2
from __future__ import annotations

import base64
import pandas as pd

# type_id -> (name, data_bytes, scale, signed)
LPP_SPEC: dict[int, tuple[str, int, float, bool]] = {
    0x02: ("analog_in",   2, 0.01, True),
    0x65: ("luminosity",  2, 1.0,  False),
    0x67: ("temperature", 2, 0.1,  True),   # 0.1 °C
    0x68: ("humidity",    1, 0.5,  False),  # 0.5 %RH
    0x73: ("pressure",    2, 0.1,  False),  # 0.1 hPa
    0x88: ("gps",         9, 1.0,  True),   # handled specially
}


def _decode_gps(chunk: bytes) -> dict[str, float]:
    """Decode a 9-byte LPP GPS field: lat/lon at 1e-4, altitude at 1e-2, big-endian signed."""
    lat = int.from_bytes(chunk[0:3], "big", signed=True) * 1e-4
    lon = int.from_bytes(chunk[3:6], "big", signed=True) * 1e-4
    alt = int.from_bytes(chunk[6:9], "big", signed=True) * 1e-2
    return {"latitude": lat, "longitude": lon, "altitude_m": alt}


def cayenne_lpp_to_df(payload_b64: str) -> pd.DataFrame:
    """Decode a Cayenne LPP base64 payload into a typed pandas DataFrame.

    Returns one row per channel with columns:
        channel (int), metric (str), value (float | NaN),
        latitude, longitude, altitude_m (float | NaN for non-GPS channels).

    Raises
    ------
    ValueError
        If the payload is not valid base64, carries an unknown type id,
        or is truncated mid-channel.
    """
    raw = base64.b64decode(payload_b64, validate=True)
    rows: list[dict] = []
    i, n = 0, len(raw)

    while i + 2 <= n:
        channel, type_id = raw[i], raw[i + 1]
        spec = LPP_SPEC.get(type_id)
        if spec is None:
            raise ValueError(f"unknown LPP type 0x{type_id:02x} at offset {i}")
        name, size, scale, signed = spec
        i += 2
        if i + size > n:
            raise ValueError(f"truncated {name} channel at offset {i}")
        chunk = raw[i:i + size]
        i += size

        row: dict = {"channel": channel, "metric": name, "value": float("nan"),
                     "latitude": float("nan"), "longitude": float("nan"),
                     "altitude_m": float("nan")}
        if type_id == 0x88:
            row.update(_decode_gps(chunk))
        else:
            row["value"] = int.from_bytes(chunk, "big", signed=signed) * scale
        rows.append(row)

    df = pd.DataFrame(
        rows,
        columns=["channel", "metric", "value", "latitude", "longitude", "altitude_m"],
    )
    return df.astype({
        "channel": "int16",
        "metric": "string",
        "value": "float64",
        "latitude": "float64",
        "longitude": "float64",
        "altitude_m": "float64",
    })

The decoder is deliberately stateless: hand it a payload, get a frame. To process a stream of uplinks, call it per message and pd.concat the results with a device_eui and received_at column added from the envelope — the same envelope handled when parsing The Things Network uplinks into PostGIS-ready rows.

Two design choices deserve a note. First, the function returns a fixed six-column schema even when a frame contains no GPS channel — the latitude, longitude, and altitude_m columns are present and filled with NaN. A stable schema across every device profile means a downstream pd.concat of a temperature-only frame and a GPS-bearing frame never raises on mismatched columns. Second, the bounds check if i + size > n before every read is what turns a truncated radio frame into a clean ValueError rather than an int.from_bytes on a short slice, which would silently produce a smaller-magnitude value. In a pipeline processing millions of uplinks, that difference is the line between a dead-letter entry you can investigate and a corrupted reading you never notice.

For very high uplink volumes, calling the decoder once per message and concatenating is adequate up to tens of thousands of frames per batch; beyond that, accumulate the row dicts across many uplinks and build a single DataFrame at the end, since pd.concat in a tight loop reallocates on every call. The decode work itself is trivially cheap — a handful of integer reads per frame — so the DataFrame construction, not the byte walk, is the throughput bottleneck.

Parameter Tuning Guide

Cayenne LPP resolution is fixed by the specification, so “tuning” here means knowing the exact width, sign, and scale to validate decoded magnitudes against your sensors.

Sensor / metric Type id Data bytes Signed Scale Physical range check
Air temperature 0x67 2 yes 0.1 −60 to 85 °C
Relative humidity 0x68 1 no 0.5 0 to 100 %RH
Barometric pressure 0x73 2 no 0.1 300 to 1100 hPa
Analog input (gas, voltage) 0x02 2 yes 0.01 sensor-specific
Luminosity 0x65 2 no 1.0 0 to 65535 lux
GPS latitude / longitude 0x88 9 yes 1e-4 within deployment bbox

For multi-sensor nodes, the channel byte — not the type — distinguishes two sensors of the same kind. Two 0x67 temperature channels on one uplink are an ambient reading and a probe reading; keep the channel column so they never collapse into one.

Verification and Testing

Pin the decoder against a hand-verified frame. The payload below encodes channel 1 temperature 27.2 °C (0x0110 = 272, ×0.1) and channel 2 humidity 51.0 %RH (0x66 = 102, ×0.5); the test asserts both the values and the DataFrame shape.

import base64
import numpy as np
import pandas as pd


def test_decodes_temperature_and_humidity():
    # 01 67 01 10  |  02 68 66
    frame = bytes([0x01, 0x67, 0x01, 0x10, 0x02, 0x68, 0x66])
    df = cayenne_lpp_to_df(base64.b64encode(frame).decode())

    assert list(df["metric"]) == ["temperature", "humidity"]
    assert df.loc[df["channel"] == 1, "value"].iloc[0] == 27.2
    assert df.loc[df["channel"] == 2, "value"].iloc[0] == 51.0
    assert len(df) == 2
    assert df["value"].dtype == np.float64


def test_negative_temperature_keeps_sign():
    # channel 3, temperature -5.0 °C -> -50 -> 0xFFCE two's complement
    frame = bytes([0x03, 0x67, 0xFF, 0xCE])
    df = cayenne_lpp_to_df(base64.b64encode(frame).decode())
    assert df["value"].iloc[0] == -5.0


def test_truncated_frame_raises():
    frame = bytes([0x01, 0x67, 0x01])  # temperature needs 2 data bytes, only 1 present
    import pytest
    with pytest.raises(ValueError, match="truncated"):
        cayenne_lpp_to_df(base64.b64encode(frame).decode())

The negative-temperature case is the one that catches real bugs: if the sign flag is dropped, 0xFFCE decodes to 65486 and the test fails loudly instead of a −5 °C reading silently becoming 6548.6 °C in production.

Gotchas

Dropping the sign bit. The single most common defect. Temperature, analog input, and GPS are two’s-complement signed; reading them unsigned turns every negative value into a large positive one. It hides in warm-weather testing and appears the first frosty night.

Assuming a fixed frame length. LPP frames vary in length by how many channels a device includes on a given uplink, which can change with sensor availability (no GPS lock, no external probe attached). Never hard-code a byte count — walk the frame and stop at the buffer end.

Confusing channel with type. Two sensors of the same type are distinguished only by the channel byte. Collapsing on metric alone silently merges an ambient and a probe temperature into one ambiguous series.

Unknown type ids treated as skippable. If you hit a type id not in your table you cannot know its data width, so you cannot safely skip it — the read cursor is now lost. Raise and dead-letter the frame; a new type id means firmware changed.