PS-10280 feature: Implement logic for search_by_timestamp mode #89
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://perconadev.atlassian.net/browse/PS-10280 Main executable can now be run in one more mode - 'search_by_timestamp'. Execute './binlog_server search_by_timestamp <config_file> <timestamp>' and get the list of all binlog files that have at least one event generated not later than the provided '<timestamp>'. <timestamp> is expected to be in ISO format, e.g. '2026-02-10T17:14:42'. In this mode, we simply initialize an instance of the 'binsrv::storage' class in 'read_only' mode, perform storage object listing and return binlog records that match the specified criteria in JSON format. The output is written to the standard output. The application immediately terminates after performing this operation. In addition, the application exit code can be used to determine whether the operation was successful. Refactored 'binsrv::storage' class - upon initialization it now stores the full list of known binlog files along with their metadata (file name, size, associated GTID set and min/max timestamp). Its constructor now takes one more parameter 'construction_mode' (which can be either 'querying_only' or 'streaming'). 'querying_only' is used in 'search_by_timestamp' mode. Class internal machinery now also keeps track of min/max timestamps found in the events stored in particular binlog files. They are now also being stored in binlog file metadata JSON file. Added new namespace 'binsrv::models' with the following classes: - 'binlog_record' - 'response_status' - 'error_response' - 'search_by_timestamp_response' which are used to form JSON responses for query operations (currently for 'search_by_timestamp' mode). Added 'binsrv::ctime_timestamp' class to represent a timestamp based on 'std::time_t' that can be easily converted to / from a string. Added 'binsrv::ctime_timestamp_range' class that represents a range (min and max values) of 'binsrv::ctime_timestamp'. 'binsrv::basic_storage_backend' interface extended with additional method 'get_object_uri()' that can generate storage URI for the given binlog file name (this functionality is used during generating JSON response for the 'search_by_timestamp' command). Both 'binsrv::filesystem_storage_backend' and 'binsrv::s3_storage_backend' give concrete implementation to this function. Added new 'binlog_streaming.search_by_timestamp' MTR test case with two combinations ('position' and 'gtid') that checks various scenarios of running Binlog Server utility in 'search_by_timestamp' mode. 'README.md' documentation file updated with another section describing the 'search_by_timestamp' mode.
percona-ysorokin
force-pushed
the
search_by_timestamp
branch
from
94c0b2c to
c70c961
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://perconadev.atlassian.net/browse/PS-10280
Main executable can now be run in one more mode - 'search_by_timestamp'. Execute './binlog_server search_by_timestamp <config_file> ' and get the list of all binlog files that have at least one event generated not later than the provided ''.
is expected to be in ISO format, e.g. '2026-02-10T17:14:42'.
In this mode, we simply initialize an instance of the 'binsrv::storage' class in 'read_only' mode, perform storage object listing and return binlog records that match the specified criteria in JSON format. The output is written to the standard output. The application immediately terminates after performing this operation. In addition, the application exit code can be used to determine whether the operation was successful.
Refactored 'binsrv::storage' class - upon initialization it now stores the full list of known binlog files along with their metadata (file name, size, associated GTID set and min/max timestamp). Its constructor now takes one more parameter 'construction_mode' (which can be either 'querying_only' or 'streaming'). 'querying_only' is used in 'search_by_timestamp' mode. Class internal machinery now also keeps track of min/max timestamps found in the events stored in particular binlog files. They are now also being stored in binlog file metadata JSON file.
Added new namespace 'binsrv::models' with the following classes:
Added 'binsrv::ctime_timestamp' class to represent a timestamp based on 'std::time_t' that can be easily converted to / from a string.
Added 'binsrv::ctime_timestamp_range' class that represents a range (min and max values) of 'binsrv::ctime_timestamp'.
'binsrv::basic_storage_backend' interface extended with additional method 'get_object_uri()' that can generate storage URI for the given binlog file name (this functionality is used during generating JSON response for the 'search_by_timestamp' command). Both 'binsrv::filesystem_storage_backend' and 'binsrv::s3_storage_backend' give concrete implementation to this function.
Added new 'binlog_streaming.search_by_timestamp' MTR test case with two combinations ('position' and 'gtid') that checks various scenarios of running Binlog Server utility in 'search_by_timestamp' mode.
'README.md' documentation file updated with another section describing the 'search_by_timestamp' mode.