GH-51368: [C++] Fix CSV reader dropping '\n' after a straddling '\r' inside a quoted field

[iliasabk]

Rationale for this change

With newlines_in_values = true, a quoted CRLF sequence split across two input buffers was silently corrupted: "…\r at the end of one buffer and \n… at the start of the next lost the \n (it stayed inside the field as \r only). Reported in #51368.

The root cause is in CSVBufferIterator, which unconditionally skipped a leading \n whenever the previous buffer ended with \r. At that layer there is no quoting context, so the code cannot distinguish a \r\n that is a physical line separator from \r\n that is the contents of an unfinished quoted field.

What changes are included in this PR?

• CSVBufferIterator no longer resolves straddling CRLF sequences (it still strips a leading UTF-8 BOM).
• BlockReader now owns the decision via trailing_cr_ + a SkipStraddlingCRLF helper: the next buffer's leading \n is skipped only when the previous buffer's trailing \r was fully consumed as a line separator. When the \r belongs to an unfinished quoted field, it is carried in partial_ (non-empty), the flag stays false, and the \n is preserved as field contents.
• The flag is updated in both SerialBlockReader (through consume_bytes, driven by how much of the buffer the parser consumed) and ThreadedBlockReader (through next_partial from the chunker), including the row-skipping paths. Empty buffers never overwrite the state.
• An initial prev_ended_cr flag is plumbed through the block-reader factories for the edge case where a header row ends exactly at a buffer boundary with \r (the post-header buffer is then empty and the next buffer starts with \n).
• New regression test TestStraddlingCRLF covering a quoted \r\n split across the boundary, a line-separator \r\n split across the boundary, and a header row ending with \r at the boundary. Registered for the serial, threaded-async and streaming readers.

Are these changes tested?

Yes — a new StraddlingCRLF test in csv/reader_test.cc covers all three cases above with a block size that forces the split. (Note: the change was developed and cross-validated against a transliteration of the reader/chunker/parser pipeline; the gtest cases should be run in CI.)

Are there any user-facing changes?

No API changes. Files with newlines_in_values enabled whose quoted fields contain \r\n straddling a read block boundary are now parsed correctly instead of losing the \n.

Closes #51368

• GitHub Issue: arrow::csv silently drops one byte when a CRLF inside a quoted field straddles a parser block boundary #51368

Was AI used for this PR?

PR code and description written by:

• AI

Reviewed before submission by:

• AI

[github-actions]

⚠️ GitHub issue #51368 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #51368 has no components, please add labels for components.

[Reranko05]

@iliasabk Can you declare the usage of AI in this PR, as given in the PR template.

[iliasabk]

@Reranko05 Good catch — declared. Both the code and the PR description were written by an AI coding agent and reviewed by an AI agent before submission (running under my account). I've added the disclosure section to the PR description. Thanks!

[github-actions]

⚠️ GitHub issue #51368 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #51368 has no components, please add labels for components.