Skip to content

[common] Release the input stream when FileIndexPredicate fails to open it - #9462

Merged
JingsongLi merged 1 commit into
apache:masterfrom
PDGGK:fix-fileindex-predicate-leak
Aug 29, 2026
Merged

[common] Release the input stream when FileIndexPredicate fails to open it#9462
JingsongLi merged 1 commit into
apache:masterfrom
PDGGK:fix-fileindex-predicate-leak

Conversation

@PDGGK

@PDGGK PDGGK commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Purpose

FileIndexPredicate opens a stream and hands it straight to the reader:

public FileIndexPredicate(Path path, FileIO fileIO, RowType fileRowType) throws IOException {
    this(fileIO.newInputStream(path), fileRowType);
    this.path = path;
}

public FileIndexPredicate(SeekableInputStream inputStream, RowType fileRowType) {
    this.reader = FileIndexFormat.createReader(inputStream, fileRowType);
}

FileIndexFormat.Reader validates the file header and rejects anything that is not an index file:

if (magic != MAGIC) {
    throw new RuntimeException("This file is not file index file.");
}

When that fires the constructor never returns, so nothing holds a reference to the stream and nobody can close it. A corrupted or truncated index file leaks a file handle per attempt, and scans retry per file.

BitmapIndexReader and BTreeIndexReader already guard the same sequence, so this makes the three agree.

Tests

FileIndexPredicateCloseTest#testFailedConstructionReleasesTheStream wraps the stream so closes are counted, feeds bytes that are not an index file, and asserts the construction fails with not file index file and the stream is still released.

Reverting the change turns it red on the close counter (Expecting AtomicInteger(0) to have value: 1).

mvn test -pl paimon-common -Dtest='org.apache.paimon.fileindex.**' — 68 tests, all passing. spotless and checkstyle clean.

…en it

FileIndexPredicate opens a stream and hands it straight to
FileIndexFormat.createReader, which rejects a file whose magic or version does
not match. The constructor then exits with the stream open and unreferenced,
so a corrupted index file leaks a file handle per attempt.

Close it on the way out, the same way BitmapIndexReader and BTreeIndexReader
guard their own construction.
@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit ca47334 into apache:master Aug 29, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants