feat(go): support a read-builder limit hint for split pruning

[jackylee-ch]

Go and C callers can't pass a row-count limit to scan planning, so a service that only wants to preview or sample the first N rows of a large table retains every split even though the core reader can already bound them.

The core ReadBuilder already stops retaining splits once the kept ones cover the limit (TableScan::apply_limit_pushdown), but that bound was unreachable from Go/C. Planning still reads the manifest entries to learn each split's row count, so the hint bounds how many splits are retained rather than avoiding all planning I/O.

This adds the paimon_read_builder_with_limit C FFI symbol and a Go ReadBuilder.WithLimit wrapper, threading the hint into both scan planning and new_read. A negative Go limit is rejected with ErrNegativeLimit: an int below zero would wrap through the unsigned C boundary to a huge value and make planning stop after the first split, silently dropping rows. It is a planning hint only — the caller still enforces the final row limit when reading.

Tests: a C-binding test plans multiple known-count splits and asserts a zero limit prunes them all, a small limit retains a strict non-empty subset, and a large limit retains every split; a Go test asserts a negative limit is rejected before the unsigned cast; the existing Go integration test checks split counts with and without a limit.

[JingsongLi]

Requirement fit: SUPPORTED, with a narrower performance claim. Exposing the existing split-limit hint to Go/C has real value, but plan_snapshot_from_lists reads the manifest entries before the positive-limit accumulator runs; the PR should not claim that it avoids all manifest/statistics I/O. Implementation: FINDINGS at e173fde0.

[P1] Reject negative Go limits before the unsigned cast (bindings/go/read_builder.go:83-87). WithLimit(-1) currently returns success and passes uintptr(-1) to Rust. In LimitPushdownAccumulator::push, self.limit as i64 then becomes -1, so the first split with a known row count satisfies the limit and later splits are omitted. A caller passing a negative config/sentinel gets silently incomplete scan results. Please reject limit < 0 (or use an unsigned public type), and add a Go→C→scan regression with multiple known-count splits.

Verification: the PR C test test_read_builder_with_limit_prunes_plan_splits passed. I added a temporary Rust probe with usize::MAX and two known-count splits; it failed on the first push, confirming early truncation. I removed the probe after reproduction, leaving the review checkout clean. git diff --check main...HEAD passed; all 14 PR CI jobs are green, but none covers the negative input.

[jackylee-ch]

Addressed both points.

Negative limit. ReadBuilder.WithLimit now rejects a negative int up front with ErrNegativeLimit, before the uintptr cast, so it can no longer wrap to a huge value and make LimitPushdownAccumulator stop after the first split. TestReadBuilderWithLimitRejectsNegative asserts the distinct ErrNegativeLimit (not ErrClosed) on a zero-value builder, which pins that the reject arm runs ahead of the closed-builder guard. It is non-vacuous: removing the guard makes the test fail with got paimon: use of closed resource.

Narrower performance claim. The WithLimit doc and the PR description no longer say the hint avoids reading statistics; planning still reads the manifest entries to learn each split's row count, so the hint only bounds how many splits are retained.

Multiple known-count splits. test_read_builder_with_limit_prunes_plan_splits now plans several splits — a tiny source.split.target-size keeps each committed file as its own split — and asserts the hint threads through the C FFI across them: a zero limit prunes them all, a small limit retains a strict non-empty subset, and a large limit retains every split. The accumulator behavior your probe hit is also covered directly at LimitPushdownAccumulator (test_incremental_limit_accumulator_stops_after_known_count_reaches_limit, test_apply_limit_pushdown_returns_all_when_limit_not_reached).

Rebased onto current main. paimon-c (83 tests) passes; go vet is clean and the non-warehouse Go tests pass; clippy -p paimon-c --all-targets -D warnings is clean.

[JingsongLi]

Thanks for fixing the negative Go limit and narrowing the planning claim. The Go regression and an additional Go → C → scan → reader probe now pass, including zero, small, large, maximum nonnegative int, rejected negative input and closed-builder handling.

One P2 input-range issue remains in the newly exposed C API, paimon_read_builder_with_limit (bindings/c/src/table.rs, around lines 657–669). It accepts all size_t values and returns success, but the core LimitPushdownAccumulator::push converts the limit with self.limit as i64. On a 64-bit platform, SIZE_MAX or any value above INT64_MAX becomes negative, so the first known-count split satisfies the limit and subsequent splits are omitted. A very large positive upper bound must not silently truncate a small table.

I extended the actual C FFI test after three real writes/commits (six rows across three splits). An ordinary large limit retains all three splits; usize::MAX is accepted but retains only one. The assertion fails on this head. A temporary control bounding the C-side value to i64::MAX passes. Please reject unsupported values before storing them, or make the accumulator compare/saturate without a lossy signed cast, with C-boundary coverage for INT64_MAX + 1 and SIZE_MAX.

Validation: all 83 existing C tests passed; the native library was rebuilt from this head and the two Go limit tests plus the additional real reader-chain probe passed without warehouse skips. go vet passed. The fixture was generated by native Rust writes; this was not a Spark compatibility run.

[jackylee-ch]

Addressed by fixing the root cause rather than only bounding the C input.

LimitPushdownAccumulator::push compared scanned_row_count >= self.limit as i64. Since limit is a usize, any value above i64::MAX (your SIZE_MAX case, or INT64_MAX + 1) turned negative and the first counted split satisfied it. It now compares as u128; scanned_row_count is always non-negative, so a limit a real row count cannot reach never early-stops. This makes the lossy cast harmless for every caller (C, Go, core), not just the one C entry point.

Coverage:

• Core: test_incremental_limit_accumulator_does_not_truncate_on_oversized_limit pushes two counted splits under usize::MAX and i64::MAX + 1 and asserts neither early-stops and all splits are kept.
• C boundary: test_read_builder_with_limit_prunes_plan_splits now also asserts paimon_read_builder_with_limit with SIZE_MAX and i64::MAX + 1 retains every one of its multiple splits.

I verified non-vacuity: restoring the as i64 cast makes SIZE_MAX truncate to the first split and both tests fail; the u128 comparison passes.

Rebased onto current main. The accumulator tests and paimon-c (83 tests, incl. the FFI plan test) pass; clippy -p paimon -p paimon-c --all-targets -D warnings is clean.