From cffa1cd70fd987bf084cf9708ad94f0ddc161054 Mon Sep 17 00:00:00 2001
From: Chukwudi Nwachukwu <theichux@gmail.com>
Date: Tue, 1 Sep 2026 12:51:15 +0100
Subject: [PATCH] Fix AudioFrameV3Wrapper::get_data() to return correct float32
 audio data
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

## Problem

AudioFrameV3Wrapper::get_data() wraps NDI's float32 planar audio data
as a uint8 numpy array with incorrect strides, making all audio
received via framesync_capture_audio_v2() unusable in Python.

Specifically:
- The returned numpy array has dtype=uint8 and shape (channels, samples)
- tobytes() returns only 1/4 of the actual audio data (1024 bytes
  instead of 4096 for a 1024-sample mono frame)
- The array's internal buffer is a COPY that is too small; reading
  past it via ctypes returns garbage (uninitialized memory)
- All samples appear as NaN or silent corruption

## Root Cause

AudioFrameV3Wrapper::get_data() uses sizeof(uint8_t) as the element
size and format_descriptor<uint8_t> as the numpy format, but the
underlying NDI audio data is always FLTP (float planar, 32-bit).

Meanwhile, AudioFrameV2Wrapper::get_data() correctly uses
sizeof(float) and format_descriptor<float> — V3 is a copy-paste error.

## The Fix

Change AudioFrameV3Wrapper::get_data() to match AudioFrameV2Wrapper:

  Before (buggy):
    size = sizeof(uint8_t)
    format_descriptor<uint8_t>
    stride: {col * size * 4, size}   // col * 4 bytes, 1 byte

  After (fixed):
    size = sizeof(float)
    format_descriptor<float>
    stride: {col * size, size}       // col * 4 bytes, 4 bytes

This changes the numpy array dtype from uint8 to float32 and fixes
the stride calculation so the full audio data is accessible.

## Why This Is Correct

1. NDI SDK guarantees FLTP format: the FourCC field on AudioFrameV3
   is always FOURCC_AUDIO_TYPE_FLTP (float planar). There are no
   other audio FourCC types in the current SDK.

2. Matches AudioFrameV2Wrapper: the V2 wrapper already handles this
   correctly. V3 should match.

3. Matches DistroAV (OBS NDI plugin): DistroAV's C++ code accesses
   p_data directly as float* with channel_stride_in_bytes offsets —
   confirming the data is float32.

4. Verified on real hardware: tested with NDI Camera Android app
   sending mono 48kHz audio. After the fix, framesync_capture_audio_v2()
   returns clean float32 data with shape (channels, samples),
   dtype=float32, no NaN values, and correct amplitude range.

## Reproduction

Before fix:
  import NDIlib as ndi
  import numpy as np
  ndi.initialize()
  recv = ndi.recv_create_v3()
  # ... connect to source ...
  fs = ndi.framesync_create(recv)
  af = ndi.framesync_capture_audio_v2(fs, 0, 0, 1024)
  print(af.data.dtype)     # uint8 (WRONG)
  print(af.data.shape)     # (1, 1024) — misleading, only 256 float32 samples fit
  print(af.data.nbytes)    # 1024 (WRONG, should be 4096)
  print(np.any(np.isnan(af.data)))  # True (WRONG)

After fix:
  print(af.data.dtype)     # float32 (CORRECT)
  print(af.data.shape)     # (1, 1024)
  print(af.data.nbytes)    # 4096 (CORRECT)
  print(np.any(np.isnan(af.data)))  # False (CORRECT)
  print(af.data.max())     # ~0.002 (real audio from phone mic)

## Impact

- Breaking change: No. The array shape stays (channels, samples).
  Only the dtype changes from uint8 to float32, which is the
  correct representation of the data.
- Backward compatibility: Code that treated the array as raw bytes
  via tobytes() will get different output, but that was already
  broken (only returned 1/4 of the data).
- Performance: No change. Same memory allocation, same data copy.
---
 src/main.cpp | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/main.cpp b/src/main.cpp
index b61ce94..b8f63a5 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -267,8 +267,8 @@ struct AudioFrameV3Wrapper {
     size_t col = inner.no_samples, row = inner.no_channels,
            size = sizeof(uint8_t);
     return py::array(py::buffer_info(inner.p_data, size,
-                                     py::format_descriptor<uint8_t>::format(),
-                                     2, {row, col}, {col * size * 4, size}));
+                                     py::format_descriptor<float>::format(),
+                                     2, {row, col}, {col * size, size})));
   }
 };