Ladybug version
v0.18.0
What operating system are you using?
Pop!_OS 22.04 LTS \n \l
What happened?
This Issue was authored by Claude Code using Sonnet 5.
Description
COPY <table> FROM <df> crashes with heap corruption (nondeterministic SIGSEGV or SIGABRT) when the DataFrame has an all-null (or heavily-null) STRING column and the row count exceeds 2113 rows committed in a single COPY ca
ll.
Inherited from Kuzu 0.11.3 — Ladybug is a fork of that codebase and still has the bug. Kuzu itself is abandoned upstream, so Ladybug is the version worth fixing.
Environment
- ladybug 0.18.0
- pandas 2.2.3
- pyarrow 19.0.0
- numpy 2.2.6
- Python 3.13 (Linux)
Are there known steps to reproduce?
Reproduction
See attached script: ladybug_copy_null_segfault_repro.py
Observed Behavior
The exact threshold and failure mode are stable and were confirmed across multiple runs:
| N |
Result |
| 2050, 2100, 2113 |
succeeds, exact row count returned |
| 2114 |
crashes |
| 2200, 2500 |
crashes |
Failure modes at N > 2113 are nondeterministic across runs but all indicate heap corruption inside the native COPY-from-DataFrame path:
Segmentation fault (core dumped)free(): invalid next size (normal) (SIGABRT)munmap_chunk(): invalid pointer (SIGABRT)
Expected behavior
COPY FROM <df> should succeed regardless of null count in a STRING column, the same as the file-based path does.
Workaround
Writing the DataFrame to a parquet file and using COPY <table> FROM '<file>.parquet' instead of COPY <table> FROM <df> avoids the bug — verified up to 10k rows (also independently re-verified at 5k rows with an all-null STRING colum
n) with the in-memory DataFrame path.
Notes on related issues
I searched existing issues and didn't find an exact duplicate, but two are adjacent and may share root cause in the native STRING handling for the DataFrame ingestion path:
Given the coincidence of STRING-column corruption reports on the same DataFrame-ingestion path, it may be worth checking whether #647 and this issue share a root cause in the string-chunk/dictionary encoding used when materializing pand
as STRING columns, even though the symptoms differ (crash vs. silent corruption).
Ladybug version
v0.18.0
What operating system are you using?
Pop!_OS 22.04 LTS \n \l
What happened?
This Issue was authored by Claude Code using Sonnet 5.
Description
COPY <table> FROM <df>crashes with heap corruption (nondeterministicSIGSEGVorSIGABRT) when the DataFrame has an all-null (or heavily-null)STRINGcolumn and the row count exceeds 2113 rows committed in a singleCOPYcall.
Inherited from Kuzu 0.11.3 — Ladybug is a fork of that codebase and still has the bug. Kuzu itself is abandoned upstream, so Ladybug is the version worth fixing.
Environment
Are there known steps to reproduce?
Reproduction
See attached script: ladybug_copy_null_segfault_repro.py
Observed Behavior
The exact threshold and failure mode are stable and were confirmed across multiple runs:
Failure modes at N > 2113 are nondeterministic across runs but all indicate heap corruption inside the native COPY-from-DataFrame path:
Segmentation fault (core dumped)free(): invalid next size (normal)(SIGABRT)munmap_chunk(): invalid pointer(SIGABRT)Expected behavior
COPY FROM <df>should succeed regardless of null count in a STRING column, the same as the file-based path does.Workaround
Writing the DataFrame to a parquet file and using
COPY <table> FROM '<file>.parquet'instead ofCOPY <table> FROM <df>avoids the bug — verified up to 10k rows (also independently re-verified at 5k rows with an all-null STRING column) with the in-memory DataFrame path.
Notes on related issues
I searched existing issues and didn't find an exact duplicate, but two are adjacent and may share root cause in the native STRING handling for the DataFrame ingestion path:
COPY ... FROM <df>+ STRING column + Linux path, but manifests as silent primary-key corruption rather than a crash, and no row-count threshold is mentioned.
StringChunkDatacan corrupt checkpointed relationship properties #658 —StringChunkData/dictionary-column corruption after checkpoint/reload: different trigger (partial string scans post-reload), not COPY-related.Given the coincidence of STRING-column corruption reports on the same DataFrame-ingestion path, it may be worth checking whether #647 and this issue share a root cause in the string-chunk/dictionary encoding used when materializing pand
as STRING columns, even though the symptoms differ (crash vs. silent corruption).