Describe the bug, including details regarding any error messages, version, and platform.
The Parquet reader uses disparate types for handling lengths of file components: most of the time int64_t, but sometimes also uint32_t or int32_t (especially when Thrift deserialization is involved, due to the Thrift C++ APIs).
This leads to fragilities and potential issues. This can be seen here:
|
if (footer_read_size >= (metadata_len + kFooterSize)) { |
where we are adding two uint32_ts without regard for potential addition overflow.
We should strive to use int64_t almost everywhere, only switching to narrower types when calling into third-party APIs that use such types.
Kudos to Ada Logics and Claude for submitting a more detailed example of invalid pointer read.
Component(s)
C++, Parquet
Describe the bug, including details regarding any error messages, version, and platform.
The Parquet reader uses disparate types for handling lengths of file components: most of the time
int64_t, but sometimes alsouint32_torint32_t(especially when Thrift deserialization is involved, due to the Thrift C++ APIs).This leads to fragilities and potential issues. This can be seen here:
arrow/cpp/src/parquet/file_reader.cc
Line 448 in 8b469b1
where we are adding two
uint32_ts without regard for potential addition overflow.We should strive to use
int64_talmost everywhere, only switching to narrower types when calling into third-party APIs that use such types.Kudos to Ada Logics and Claude for submitting a more detailed example of invalid pointer read.
Component(s)
C++, Parquet