flatbuffers: Add opt-in buffer identification mechanism#174
Draft
OliverHeilwagen wants to merge 2 commits into
Draft
flatbuffers: Add opt-in buffer identification mechanism#174OliverHeilwagen wants to merge 2 commits into
OliverHeilwagen wants to merge 2 commits into
Conversation
OliverHeilwagen
temporarily deployed
to
workflow-approval
GitHub Actions
Inactive
OliverHeilwagen
temporarily deployed
to
workflow-approval
GitHub Actions
Inactive
|
The created documentation from the pull request is available at: docu-html |
OliverHeilwagen
commented
May 5, 2026
| doc = "Injects major/minor version fields into a JSON file for use with serialize_versioned_buffer.", | ||
| ) | ||
|
|
||
| def serialize_versioned_buffer( |
Contributor
Author
There was a problem hiding this comment.
@4og @paulquiring
I introduced the following rules separately so we can have the major/minor version as mandatory rule parameter:
- serialize_versioned_buffer
- serialize_multiple_versioned_buffer
I am not so happy with the longer name but kept it for now to avoid breaking changes.
Since we are still early with the flatbuffers rollout we could do a breaking change of the rule name and have the following ones that should remain stable:
- serialize_buffer
- serialize_buffers (breaking former: serialize_multiple_buffers)
- serialize_versioned_buffer
- serialize_versioned_buffers
The buffer version have dedicated data and output parameters.
The buffers version have data_dict parameter (dictionary key: json data, value: output name).
Please let me know if you have any preferences.
OliverHeilwagen
changed the title
OliverHeilwagen
force-pushed
the
flatbuffers_common_buffer_identification
branch
from
69ee405 to
14332c1
Compare
OliverHeilwagen
temporarily deployed
to
workflow-approval
GitHub Actions
Inactive
OliverHeilwagen
temporarily deployed
to
workflow-approval
GitHub Actions
Inactive
OliverHeilwagen
force-pushed
the
flatbuffers_common_buffer_identification
branch
from
14332c1 to
44269a7
Compare
OliverHeilwagen
requested a deployment
to
workflow-approval
GitHub Actions
Waiting
OliverHeilwagen
requested a deployment
to
workflow-approval
GitHub Actions
Waiting
OliverHeilwagen
force-pushed
the
flatbuffers_common_buffer_identification
branch
from
44269a7 to
f68dbf7
Compare
OliverHeilwagen
temporarily deployed
to
workflow-approval
GitHub Actions
Inactive
OliverHeilwagen
temporarily deployed
to
workflow-approval
GitHub Actions
Inactive
Introduces major/minor version injection into FlatBuffer binary configuration files at build time, enabling universal buffer identification at runtime. - New serialize_versioned_buffer and serialize_multiple_versioned_buffers Bazel macros that patch version fields into JSON before flatc serialization - New inject_buffer_version.py script and Bazel rule that injects major/minor version into a JSON data file as a build action - Added BufferVersion schema under score/flatbuffers/common as shared include for versioned schemas - Added BufferVersionEnvelope schema under score/flatbuffers/common used internally to implement GetBufferVersion and VerifyBufferVersion without hardcoded vtable offsets - Extended existing serialize_buffer, serialize_multiple_buffers, generate_cpp and generate_json_schema rules with an includes attribute to support schema include dependencies - Added Starlark rule tests covering single/multiple versioned buffers and fault injection cases (out-of-range versions, wrong version_info placement)
Introduces IVersionReader and VersionReader to read and verify version metadata from FlatBuffer binaries at runtime using the universal buffer identification convention established by the BufferVersionEnvelope schema. - New BufferVersionInfo value type holding a 4-char file identifier plus major_version and minor_version fields extracted from a conforming buffer - New IVersionReader interface with GetVersion and VerifyVersion methods, supporting kExact and kMinorMinimum match modes - New VersionReader concrete implementation backed by FlatBuffers Verifier for structural integrity checking before field access - New ErrorCode error domain covering kNullDataPointer, kVerificationFailed, kVersionInfoNotPresent, and kVersionMismatch failure cases - Updated buffer_version.fbs and buffer_version_envelope.fbs schemas to align with the new convention and added type_marker to enable misplacement check - Added integration tests covering nominal reads, exact/minimum version matching, out-of-range versions, wrong-field placement, and null inputs
OliverHeilwagen
force-pushed
the
flatbuffers_common_buffer_identification
branch
from
f68dbf7 to
d51fc10
Compare
OliverHeilwagen
requested a deployment
to
workflow-approval
GitHub Actions
Waiting
OliverHeilwagen
requested a deployment
to
workflow-approval
GitHub Actions
Waiting
OliverHeilwagen
commented
Jun 3, 2026
|
|
||
| TEST_F(VersionReaderTest, GetVersionRejectsNullPointer) | ||
| { | ||
| RecordProperty("PartiallyVerifies", "comp_req__flatbuffers__buffer_identification"); |
Contributor
Author
There was a problem hiding this comment.
Check if process is considering multiple PartiallyVerifies as FullyVerifies, or if the test cases should already claim FullyVerifies.
OliverHeilwagen
commented
Jun 3, 2026
| @@ -0,0 +1,38 @@ | |||
| /******************************************************************************** | |||
Contributor
Author
There was a problem hiding this comment.
Correct name to buffer_version_info.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
resolves #128
Todo: