fix(etl): upsert explicit subscription writes#335
Merged
Conversation
raymondjacobson
changed the title
social_subscription.go was the one social handler #331 left on demote-then-insert; follow/repost/save were converted to ON CONFLICT upserts. subscriptions is also the only social table with two writers (explicit Subscribe + Follow auto-subscribe), and demote-then-insert is a two-statement write: between the demote and the insert the other writer can land a second is_current row. With no uniqueness constraint, that accumulated the 92 duplicate current rows that later failed the 0030 index build. Convert insertSubscription to the same single-statement ON CONFLICT upsert as the Follow path so the arbiter index fully enforces one-current-row-per- identity for both writers and dupes can't recur. The migration dedupe remains required to clean pre-existing rows so the unique index can build. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raymondjacobson
force-pushed
the
fix/etl-0030-dedupe-current
branch
from
128fe7e to
6d39f11
Compare
raymondjacobson
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
subscriptionsis the only one of the four social tables (reposts/saves/follows/subscriptions) with two independent writers:social_subscription.go→ demote-then-insert (UPDATE … is_current=falsethen a plainINSERT).social_follow.go→ON CONFLICTupsert.The other three tables have a single writer each. Demote-then-insert is a two-statement write: between the
UPDATEand theINSERT, the other subscription writer can land a secondis_currentrow — and the table had no uniqueness constraint to stop it. Over time that accumulated duplicateis_currentrows insubscriptions(92 of them in prod), which the partial unique index added in #331 (subscriptions_current_uniq_idx) was not equipped to coexist with.#331 (
24084d2) convertedfollow/repost/saveto upserts and added the four arbiter indexes, but never touchedsocial_subscription.go(last modified by an unrelated rename in #300). So subscriptions got an arbiter index while one of its two writers still used the demote-then-insert pattern that can violate it.Fix
Convert
insertSubscriptionto the same single-statementON CONFLICTupsert used by the Follow path. Both subscription writers now go through the arbiter atomically, so the unique index fully enforces one-current-row-per-identity and duplicates can't recur.Scope
Code-only. The pre-existing duplicate rows were already cleaned up out-of-band in the single deployment that had them, and golang-migrate tracks version number only (it won't re-run
0030), so no migration-side dedupe/backfill is included here —0030is unchanged.Test plan
go build ./...andgo vetinpkg/etlgo test ./processors/entity_manager/(DB-backed: runs migrations + exercises subscribe/follow/contest paths) — pass🤖 Generated with Claude Code