Repository navigation
Look up accounting reservations by primary key from the usage trigger - #41
Merged
Merged
Conversation
The usage trigger passes TG_TABLE_NAME (collation "C") down to weave_reserve_refresh and weave_allocation_delta, so their kind and relation comparisons could not use the default-collation primary keys and every charged write scanned all reservations and allocations. Migration 0031_operations_key_collation patches both shared functions to compare in the default collation; an auto_explain integration test pins the primary-key plans and unchanged usage accounting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Starting a run got slower with every run ever started: the usage accounting trigger looked up its reservation by scanning every row of
operation_reservations(36–90 ms per inserted run at 524,000 reservations).Cause.
weave_usage_chargepassesTG_TABLE_NAME(typename, collation"C") down the call chain, so PL/pgSQL gives every text parameter and variable ofweave_reservation_touch,weave_reserve_refreshandweave_allocation_deltacollation"C". Compared with the default-collation key columns, that rules out the primary key. auto_explain inside the trigger shows three affected statements:weave_reserve_refreshSELECT amount,outbox_slots FROM operation_reservations WHERE kind=kind_name AND resource_id=identifierkind = 'run'::text COLLATE "C"weave_reserve_refreshEXISTS(... operation_control_context ... AND kind=kind_name ...)COLLATE "C"key conditionweave_allocation_deltaSELECT/DELETE ... operation_allocations WHERE relation=relation_name AND row_key=identity_keyCOLLATE "C"weave_control_begin/weave_control_finishreceive literals or table-readkindvalues and are not affected.Fix. Forward migration
0031_operations_key_collationpatches both shared functions in place (pg_get_functiondef+replace, as in 0021/0025), comparing in the default collation. It fails closed unless each pattern occurs exactly twice;CREATE OR REPLACEkeeps owner, grants andSEToptions. No table, data or role change.SCHEMA_VERSIONmoves to the new head.Tests
tests/integration/test_reservation_lookup.py: seeds 6,000 reservations and 6,000 allocations in another project, inserts a run through the trigger with auto_explain, and asserts index scans onoperation_reservations_pkey/operation_allocations_pkeywith both key columns as index conditions, noCOLLATE "C"condition on any of the three tables, and unchanged usage accounting (assert_usagebefore and after).tests/integration/operations_support.pycarriesrecompute_usage/assert_usageverbatim fromfeat/operations-observability, which is not merged yet; that branch's file is a superset.make check: all requested checks passed.colima-weave-tests):test_reservation_lookup,test_operations,test_large_row_accounting,test_run_lifecycle,test_outbox,test_terminal_capacity,test_boot,test_incident_migration,test_schedule_migration,test_runs,test_waits: 115 passed.Upgrade note
Operators must run the explicit migration before restarting the server; CHANGELOG (Unreleased) and
docs/operations/upgrades.mdare updated.Coordination
feat/operations-observabilityalso adds a0031_*migration. Whichever merges second renumbers to0032_*and chains itsdown_revision.