rkdecoder-python

Python package for media codecs available for Rockchip

rkdecoder-python is a Python package built on top of Rockchip MPP, Rockchip RGA, FFmpeg, and pybind11.

It provides a Python-friendly interface for:

• decoding RTSP video streams with Rockchip hardware decoding
• optional image resize with RGA
• optional color conversion to formats such as rgb, bgr, rgba, bgra, or nv12
• optional zero-copy output through DMA-backed buffers

The package is designed for Rockchip Linux systems where MPP and RGA are available.

---

Features

• Hardware-accelerated RTSP decoding using Rockchip MPP
• Optional resize using Rockchip RGA
• Optional color conversion with RGA
• Optional zero-copy output as a DMA-backed Python object
• Python API via pybind11
• Multiple decoder instances supported
• Suitable for integration with downstream inference or vision pipelines

---

Requirements

Runtime / system dependencies

You need the following components installed on your Rockchip device:

• Rockchip MPP development headers and library

• Rockchip RGA development headers and library

• FFmpeg development headers and libraries:

  • libavformat
  • libavcodec
  • libavutil

• Python 3

• pybind11

• CMake

• a C++17 compiler

Notes

• MPP and RGA installation paths vary between devices and SDKs.
• This project is intentionally configurable through CMake -D... options so users can point to their local installation paths.
• On some boards there may be multiple FFmpeg header trees installed. In that case, you may need to force the FFmpeg include/library paths explicitly.

---

Build

Basic build

mkdir -p build
cmake -S . -B build
cmake --build build -j

After building, the Python package will be available under:

build/python

You can then import it by adding build/python to PYTHONPATH, or by passing that directory in your test scripts.

---

Configurable CMake options

The build is designed to be path-configurable.

Python / pybind11

• Python3_EXECUTABLE
• pybind11_DIR

Rockchip MPP

• MPP_ROOT
• MPP_INCLUDE_HINTS
• MPP_LIB_HINTS

Rockchip RGA

• RGA_ROOT
• RGA_INCLUDE_HINTS
• RGA_LIB_HINTS

FFmpeg

• FFMPEG_ROOT
• FFMPEG_INCLUDE_HINTS
• FFMPEG_LIB_HINTS

Package layout

• RKDECODER_PYTHON_PACKAGE_SUBDIR

---

Example build commands

1. Standard system installation

cmake -S . -B build \
  -DPython3_EXECUTABLE=$(which python)
cmake --build build -j

2. Non-standard MPP / RGA installation

cmake -S . -B build \
  -DPython3_EXECUTABLE=$(which python) \
  -DMPP_INCLUDE_HINTS=/usr/local/include \
  -DMPP_LIB_HINTS=/usr/local/lib \
  -DRGA_INCLUDE_HINTS="/path/to/rkrga/include;/path/to/rkrga/im2d_api" \
  -DRGA_LIB_HINTS=/path/to/rkrga_build
cmake --build build -j

3. Force a specific FFmpeg installation

cmake -S . -B build \
  -DPython3_EXECUTABLE=$(which python) \
  -DFFMPEG_INCLUDE_HINTS="/opt/ffmpeg/include;/opt/ffmpeg/include/aarch64-linux-gnu" \
  -DFFMPEG_LIB_HINTS="/opt/ffmpeg/lib;/opt/ffmpeg/lib/aarch64-linux-gnu" \
  -DMPP_INCLUDE_HINTS=/usr/local/include \
  -DMPP_LIB_HINTS=/usr/local/lib \
  -DRGA_INCLUDE_HINTS="/path/to/rkrga/include;/path/to/rkrga/im2d_api" \
  -DRGA_LIB_HINTS=/path/to/rkrga_build
cmake --build build -j

4. Explicit pybind11 CMake path

cmake -S . -B build \
  -DPython3_EXECUTABLE=$(which python) \
  -Dpybind11_DIR=$(python -m pybind11 --cmakedir)
cmake --build build -j

---

Importing the package

If you built the project locally, the package is typically located under:

build/python

Use it like this:

PYTHONPATH=build/python python

Then in Python:

import rkdecoder

---

Python API

Create a decoder

import rkdecoder

decoder = rkdecoder.rkdecoder(
    "rtsp://127.0.0.1:8554/mystream",
    output_format="rgb",
    size=(640, 640),
    zero_copy=False,
)

Parameters

• url: RTSP URL

• output_format: one of:

  • "rgb"
  • "bgr"
  • "rgba"
  • "bgra"
  • "nv12"

• size: optional output size as (width, height)

  • use None to disable resize

• zero_copy: whether to return DMA-backed output instead of a NumPy array

Get a frame

frame = decoder.get(timeout_ms=1000)

---

Example

Decode RTSP to RGB without resize

import rkdecoder

decoder = rkdecoder.rkdecoder(
    "rtsp://127.0.0.1:8554/mystream",
    output_format="rgb",
    size=None,
    zero_copy=False,
)

while True:
    frame = decoder.get(timeout_ms=1000)
    print(frame.shape)

Decode RTSP to RGB with resize

import rkdecoder

decoder = rkdecoder.rkdecoder(
    "rtsp://127.0.0.1:8554/mystream",
    output_format="rgb",
    size=(640, 640),
    zero_copy=False,
)

frame = decoder.get(timeout_ms=1000)
print(frame.shape)

Zero-copy decode

import rkdecoder

decoder = rkdecoder.rkdecoder(
    "rtsp://127.0.0.1:8554/mystream",
    output_format="rgb",
    size=(640, 640),
    zero_copy=True,
)

frame = decoder.get(timeout_ms=1000)
print(frame.fd, frame.width, frame.height)

---

Benchmark / test script

A Python benchmark script can be placed under test/, for example:

python test/benchmark.py \
  "rtsp://127.0.0.1:8554/mystream" \
  --python-package-dir build/python \
  --size 640x640 \
  --output-format rgb

Example with zero-copy:

python test/benchmark.py \
  "rtsp://127.0.0.1:8554/mystream" \
  --python-package-dir build/python \
  --size 640x640 \
  --output-format rgb \
  --zero-copy

---

Troubleshooting

1. pybind11 not found

Install pybind11 in the Python environment used for build:

python -m pip install pybind11

Or pass:

-Dpybind11_DIR=$(python -m pybind11 --cmakedir)

---

2. Could NOT find Python3 (missing: Development.Module)

Some older CMake versions behave differently. Use a CMake version that properly supports your Python toolchain, or ensure Python development headers are installed.

---

3. FFmpeg header conflicts

If your system contains both:

• /usr/include/libavcodec
• /usr/include/<triplet>/libavcodec

you may need to force a single FFmpeg tree:

-DFFMPEG_INCLUDE_HINTS=...
-DFFMPEG_LIB_HINTS=...

---

4. rga.h: No such file or directory

Your RGA installation may split headers across multiple directories, commonly:

• include
• im2d_api

Pass both:

-DRGA_INCLUDE_HINTS="/path/to/rkrga/include;/path/to/rkrga/im2d_api"

---

5. no usable dma-heap device found

Check:

• /dev/dma_heap/*
• /dev/dri/renderD*

This may be caused by:

• missing allocator devices
• permission problems
• environment-specific DMA allocation differences

---

6. Import error caused by Python ABI mismatch

If the built module name contains a Python ABI tag that does not match your runtime Python, rebuild using the exact Python interpreter you plan to run with:

-DPython3_EXECUTABLE=$(which python)

---

Limitations

• This project targets Rockchip Linux systems
• MPP / RGA behavior may vary between SoCs, SDK versions, and distro packaging
• Zero-copy output depends on allocator/device availability
• Some performance characteristics depend on stream format, driver versions, and memory path