Skip to content

refactor(storage): move the gateway's four stores into submitqueue/gateway/extension/storage - #741

Merged
behinddwalls merged 1 commit into
mainfrom
chenghan.ying/sq-storage-1-gateway
Sep 25, 2026
Merged

behinddwalls merged 1 commit into
mainfrom
chenghan.ying/sq-storage-1-gateway

Conversation

@roychying

@roychying roychying commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Why?

First half of the split. The gateway resolves exactly four stores — the append-only request log and the three read models — and the orchestrator the other nine, with no overlap in either direction.

What?

submitqueue/gateway/extension/storage/ gains its own Factory and a four-accessor Storage, plus the four MySQL implementations, their tests and their four .sql files. The old aggregate drops to nine accessors.

The thirteen store contracts and their mocks stay at submitqueue/extension/storage/. That package keeps its import path, so callers naming only a contract or an error are untouched.

Also adds a union schema target (//submitqueue/extension/storage:schema) plus a testutil helper, so colocated callers name no service directory, and narrows each per-service integration suite to its own schema.

Two repo-level lists needed the new package, neither discoverable from the code: the mocks Makefile target and queueshard's schemaRoots.

Test Plan

  • go build ./..., go vet ./..., make lint, make check-mocks, make check-gazelle, make check-tidy all clean; make test passes 121/121.
  • queueshard still reports 25 tables.

Issue

@roychying
roychying added this pull request to stack #742 September 22, 2026 23:11

@behinddwalls behinddwalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-written.

The service aggregate/schema split is right. These comments cover the remaining differences from the proposed structure.

Comment thread submitqueue/gateway/extension/storage/storage.go Outdated
Comment thread submitqueue/core/request/materializer.go
Comment thread test/integration/submitqueue/orchestrator/suite_test.go Outdated
Comment thread submitqueue/gateway/extension/storage/mysql/schema/BUILD.bazel

@behinddwalls behinddwalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-written.

Requesting changes: the 4-accessor gateway aggregate and schema split are right, but store interfaces should stay in submitqueue/extension/storage/. core/ importing gateway storage is the inversion to avoid. Also drop gateway schema from the orchestrator suite and add a union schema target for colocated deploys.

Comment thread submitqueue/core/request/request.go Outdated
@behinddwalls
behinddwalls force-pushed the chenghan.ying/sq-storage-1-gateway branch from e1b1cc8 to d13b22b Compare September 23, 2026 15:02
@roychying
roychying force-pushed the chenghan.ying/sq-storage-1-gateway branch from d13b22b to 52bb2aa Compare September 23, 2026 17:24
@roychying
roychying force-pushed the chenghan.ying/sq-storage-1-gateway branch from 52bb2aa to 5806046 Compare September 23, 2026 19:10
@roychying
roychying force-pushed the chenghan.ying/sq-storage-1-gateway branch from 5806046 to d638ec2 Compare September 23, 2026 19:22
Base automatically changed from chenghan.ying/service-scoped-extensions-rfc to main September 25, 2026 01:39
…teway/extension/storage

# Conflicts:
#	Makefile

# Please enter the commit message for your changes. Lines starting
# with '#' will be kept; you may remove them yourself if you want to.
# An empty message aborts the commit.
#
# Author:    Chenghan Ying <125171961+roychying@users.noreply.github.com>
#
# interactive rebase in progress; onto b7aefe7
# Last command done (1 command done):
#    pick d638ec2 # refactor(storage): move the gateway's four stores into submitqueue/gateway/extension/storage
# No commands remaining.
# You are currently rebasing branch 'chenghan.ying/sq-storage-1-gateway' on 'b7aefe7a'.
#
# Changes to be committed:
#	modified:   Makefile
#	modified:   service/submitqueue/gateway/server/BUILD.bazel
#	modified:   service/submitqueue/gateway/server/main.go
#	modified:   submitqueue/core/request/BUILD.bazel
#	modified:   submitqueue/core/request/materializer.go
#	modified:   submitqueue/core/request/materializer_test.go
#	modified:   submitqueue/core/request/request.go
#	modified:   submitqueue/extension/storage/BUILD.bazel
#	modified:   submitqueue/extension/storage/mock/storage_mock.go
#	modified:   submitqueue/extension/storage/mysql/BUILD.bazel
#	modified:   submitqueue/extension/storage/mysql/storage.go
#	modified:   submitqueue/extension/storage/mysql/storage_test.go
#	modified:   submitqueue/extension/storage/storage.go
#	modified:   submitqueue/gateway/controller/BUILD.bazel
#	modified:   submitqueue/gateway/controller/cancel.go
#	modified:   submitqueue/gateway/controller/land.go
#	modified:   submitqueue/gateway/controller/land_test.go
#	modified:   submitqueue/gateway/controller/list.go
#	modified:   submitqueue/gateway/controller/list_test.go
#	modified:   submitqueue/gateway/controller/log/BUILD.bazel
#	modified:   submitqueue/gateway/controller/log/log_test.go
#	modified:   submitqueue/gateway/controller/request_history.go
#	modified:   submitqueue/gateway/controller/request_history_test.go
#	modified:   submitqueue/gateway/controller/request_summary.go
#	modified:   submitqueue/gateway/controller/request_summary_test.go
#	modified:   submitqueue/gateway/controller/storage_fixture_test.go
#	new file:   submitqueue/gateway/extension/storage/BUILD.bazel
#	new file:   submitqueue/gateway/extension/storage/mock/BUILD.bazel
#	new file:   submitqueue/gateway/extension/storage/mock/storage_mock.go
#	new file:   submitqueue/gateway/extension/storage/mysql/BUILD.bazel
#	renamed:    submitqueue/extension/storage/mysql/request_log_store.go -> submitqueue/gateway/extension/storage/mysql/request_log_store.go
#	renamed:    submitqueue/extension/storage/mysql/request_log_store_test.go -> submitqueue/gateway/extension/storage/mysql/request_log_store_test.go
#	renamed:    submitqueue/extension/storage/mysql/request_queue_summary_store.go -> submitqueue/gateway/extension/storage/mysql/request_queue_summary_store.go
#	renamed:    submitqueue/extension/storage/mysql/request_queue_summary_store_test.go -> submitqueue/gateway/extension/storage/mysql/request_queue_summary_store_test.go
#	renamed:    submitqueue/extension/storage/mysql/request_summary_store.go -> submitqueue/gateway/extension/storage/mysql/request_summary_store.go
#	renamed:    submitqueue/extension/storage/mysql/request_summary_store_test.go -> submitqueue/gateway/extension/storage/mysql/request_summary_store_test.go
#	renamed:    submitqueue/extension/storage/mysql/request_uri_store.go -> submitqueue/gateway/extension/storage/mysql/request_uri_store.go
#	renamed:    submitqueue/extension/storage/mysql/request_uri_store_test.go -> submitqueue/gateway/extension/storage/mysql/request_uri_store_test.go
#	new file:   submitqueue/gateway/extension/storage/mysql/schema/BUILD.bazel
#	renamed:    submitqueue/extension/storage/mysql/schema/change_uri_request_mapping.sql -> submitqueue/gateway/extension/storage/mysql/schema/change_uri_request_mapping.sql
#	renamed:    submitqueue/extension/storage/mysql/schema/request_log.sql -> submitqueue/gateway/extension/storage/mysql/schema/request_log.sql
#	renamed:    submitqueue/extension/storage/mysql/schema/request_summary.sql -> submitqueue/gateway/extension/storage/mysql/schema/request_summary.sql
#	renamed:    submitqueue/extension/storage/mysql/schema/request_summary_by_queue.sql -> submitqueue/gateway/extension/storage/mysql/schema/request_summary_by_queue.sql
#	new file:   submitqueue/gateway/extension/storage/mysql/storage.go
#	new file:   submitqueue/gateway/extension/storage/mysql/storage_test.go
#	new file:   submitqueue/gateway/extension/storage/storage.go
#	modified:   test/e2e/submitqueue/BUILD.bazel
#	modified:   test/e2e/submitqueue/git_suite_test.go
#	modified:   test/e2e/submitqueue/suite_test.go
#	modified:   test/integration/submitqueue/extension/storage/BUILD.bazel
#	modified:   test/integration/submitqueue/extension/storage/mysql/BUILD.bazel
#	modified:   test/integration/submitqueue/extension/storage/mysql/storage_test.go
#	modified:   test/integration/submitqueue/extension/storage/suite.go
#	modified:   test/integration/submitqueue/gateway/BUILD.bazel
#	modified:   test/integration/submitqueue/gateway/suite_test.go
#	modified:   test/testutil/schema.go
#	modified:   tool/linter/queueshard/main.go
#
@behinddwalls
behinddwalls force-pushed the chenghan.ying/sq-storage-1-gateway branch from d638ec2 to 4e7e0ed Compare September 25, 2026 02:01
@behinddwalls
behinddwalls added this pull request to the merge queue Sep 25, 2026
Merged via the queue into main with commit 08a077a Sep 25, 2026
16 checks passed
@behinddwalls
behinddwalls deleted the chenghan.ying/sq-storage-1-gateway branch September 25, 2026 02:13

This branch was successfully deployed

1 active deployment
stack-rebase — 4e7e0edf Deployed Sep 25, 2026 by behinddwalls via Rebase Stack #537
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants