-
Notifications
You must be signed in to change notification settings - Fork 55
240 lines (225 loc) · 11.6 KB
/
Copy pathpropagate_release_bundle.yml
File metadata and controls
240 lines (225 loc) · 11.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
name: Propagate release bundle
# On repository_dispatch (event type publish-bundle, sent by
# sync_code_repo.yml in code-development right after it publishes a release
# here), download the release asset simplerisk-<version>.tgz and propagate
# those exact bytes to the legacy distribution channels:
# 1. simplerisk/bundles (master) - via a real git push from a BLOBLESS,
# UNCHECKED-OUT clone (--filter=blob:none --no-checkout). The repo is
# ~3.4 GB of historical tarballs, so a plain or depth-1 clone fetches every
# one of them; --no-checkout is what makes the filter hold, since without it
# materialising the working tree lazily fetches them anyway. This replaced a
# git/blobs POST that could no longer carry the bundle - see the step below.
# 2. s3://<DOWNLOADS_BUCKET>/public/bundles/ - unconditional overwrite;
# a re-release means the old object is stale by definition
# Both converge on re-runs; heal anything by re-dispatching. The prod
# releases.xml bundle_md5/sha256 are NOT written here - sync_code_repo.yml
# (which builds the asset) writes the complete prod feed entry with the
# real hashes; the S3 object above derives from the same asset, so they
# agree by construction.
#
# Explicit dispatch (not on: release: published) is deliberate: it fires
# only after the asset exists, and it covers re-releases - a
# `gh release upload --clobber` fires no release event, which is exactly
# the case where stale channel artifacts must be replaced.
#
# Requires (one-time setup):
# - Secrets CORE_SYNC_APP_ID / CORE_SYNC_APP_PRIVATE_KEY (the
# SimpleRisk Core Sync App, installed on code, bundles, and
# updates.simplerisk.com)
# - Variables AWS_REGION, DOWNLOADS_BUCKET, DOWNLOADS_PUBLISHER_ROLE_ARN
# (OIDC role trusting this repo's master, PutObject on
# public/bundles/*)
#
# Design: docs/superpowers/specs/2026-06-06-release-bundle-propagation-design.md
# in the code-development repo.
on:
repository_dispatch:
types: [publish-bundle]
# Manual heals and ad-hoc propagation of backfilled releases.
workflow_dispatch:
inputs:
version:
description: 'Release version (YYYYMMDD-NNN)'
required: true
permissions:
contents: read
id-token: write # OIDC role assumption for the S3 upload
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
# Re-dispatches for the same or different versions queue rather than race.
concurrency:
group: propagate-release-bundle
cancel-in-progress: false
jobs:
propagate:
# The CORE_SYNC App credentials are environment secrets on `release`,
# whose deployment-branch policy pins it to master -- the ref both real
# triggers already use (repository_dispatch always runs on the default
# branch, and the workflow_dispatch heal reads a release asset from it).
environment: release
runs-on: ubuntu-latest
steps:
- name: Resolve and validate version
id: ver
env:
DISPATCH_VERSION: ${{ github.event.client_payload.version }}
INPUT_VERSION: ${{ inputs.version }}
run: |
set -euo pipefail
VERSION="${DISPATCH_VERSION:-$INPUT_VERSION}"
if ! printf '%s' "$VERSION" | grep -qE '^[0-9]{8}-[0-9]{3}$'; then
echo "::error::invalid or missing version: '$VERSION'"; exit 1
fi
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
- name: Download release asset and compute checksums
id: asset
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ steps.ver.outputs.version }}
run: |
set -euo pipefail
gh release download "$VERSION" -R simplerisk/code \
--pattern "simplerisk-$VERSION.tgz" --output "simplerisk-$VERSION.tgz"
SIZE=$(wc -c < "simplerisk-$VERSION.tgz" | tr -d ' ')
SIZE_MB=$((SIZE / 1048576))
# 100 MB is the PUSH limit. It is NOT the git/blobs API limit, which is
# far lower and undocumented: a 60.2 MiB bundle POSTed fine for
# 20260811-001 and 61.2 MiB failed with HTTP 422 "your input was too
# large to process" for 20260820-001. This guard used to refuse at 95 MiB
# "too close to the 100 MB API/git limit" while the write path below
# POSTed a blob -- so it waved through every size that actually fails.
# The write path now pushes, which is what makes 100 MB the real ceiling
# and this guard meaningful. Thresholds match publish-bundle.yml's RC
# guard in code-development deliberately: the two bundles writers
# diverging invisibly is what produced the incident.
echo "bundle size: ${SIZE_MB} MB (push limit 100 MB, hard guard 90 MB)" >> "$GITHUB_STEP_SUMMARY"
if [ "$SIZE" -gt 94371840 ]; then
echo "::error::bundle is ${SIZE_MB} MB — above the 90 MB guard. GitHub hard-rejects a pushed file over 100 MB, so the bundles@master archive is about to become impossible. Shrink the bundle or move the archive off git."
exit 1
elif [ "$SIZE" -gt 73400320 ]; then
REMAINING_MB=$((90 - SIZE_MB))
echo "::warning::bundle is ${SIZE_MB} MB — past 70 MB, and ${REMAINING_MB} MB under the 90 MB hard guard (~${REMAINING_MB} releases at the observed ~1 MB/release). Plan the shrink or the move off git now."
fi
MD5=$(md5sum "simplerisk-$VERSION.tgz" | cut -d' ' -f1)
SHA256=$(sha256sum "simplerisk-$VERSION.tgz" | cut -d' ' -f1)
{
echo "md5=$MD5"
echo "sha256=$SHA256"
echo "size=$SIZE"
} >> "$GITHUB_OUTPUT"
# SHA-pinned (same pin as sync_code_repo.yml in code-development) so a
# supply-chain compromise of the action repo can't run new code in a
# token-minting job.
- name: Mint bundles-repo token
id: bundles-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.CORE_SYNC_APP_ID }}
private-key: ${{ secrets.CORE_SYNC_APP_PRIVATE_KEY }}
owner: simplerisk
repositories: bundles
- name: Push bundle to the bundles repo
id: bundles
env:
GH_TOKEN: ${{ steps.bundles-token.outputs.token }}
VERSION: ${{ steps.ver.outputs.version }}
run: |
set -euo pipefail
FILE="simplerisk-$VERSION.tgz"
# Idempotency: compare git blob shas via the trees API (the
# contents API errors on >1 MB files; tree listings are cheap).
NEW_SHA=$(git hash-object "$FILE")
# An absent file yields empty jq output with exit 0; a non-zero
# exit is a REAL API failure (auth, rate limit, network) and must
# fail here, not several Git Data calls later with a cryptic error.
EXISTING_SHA=$(gh api "repos/simplerisk/bundles/git/trees/master" \
--jq ".tree[] | select(.path==\"$FILE\") | .sha") \
|| { echo "::error::failed to fetch bundles tree — cannot check idempotency"; exit 1; }
if [ "$NEW_SHA" = "$EXISTING_SHA" ]; then
echo "action=identical-skip" >> "$GITHUB_OUTPUT"
exit 0
fi
# A real git push from a BLOBLESS, UNCHECKED-OUT partial clone -- NOT the
# git/blobs API.
#
# This step used to POST the tarball to git/blobs, whose request-body
# ceiling is far below the 100 MB per-file PUSH limit its own comment
# cited. code-development's RC archive hit that ceiling on 20260820-001
# (61.2 MiB -> HTTP 422) after 60.2 MiB had worked the release before, and
# this workflow writes the SAME artifact to bundles@master, so it was
# guaranteed to fail identically at GA. Fixed here the same way.
#
# --filter=blob:none alone is not enough: without --no-checkout,
# materialising the working tree lazily fetches every blob at HEAD -- 2.09 GB
# across 110 tarballs. With it nothing is materialised, the index is read
# straight from the commit tree (read-tree needs blob SHAs, not content),
# and the push sends only the new blob, tree and commit.
#
# Not --depth 1: pushing from a shallow clone is unreliable, and blobless
# history here is metadata only.
BUNDLE_ABS="$(pwd)/$FILE"
rm -rf /tmp/bundles-ga
# The clone URL embeds the App token and git persists it in .git/config.
# Under `set -euo pipefail` any later failure would skip a trailing
# cleanup, so removal is a trap rather than a last line.
trap 'rm -rf /tmp/bundles-ga' EXIT
git clone --filter=blob:none --no-checkout --single-branch --branch master \
"https://x-access-token:${GH_TOKEN}@github.com/simplerisk/bundles.git" \
/tmp/bundles-ga
cd /tmp/bundles-ga
git config user.name "SimpleRisk Updater"
git config user.email "support@simplerisk.com"
# Assert the filter held. DIAGNOSTIC, not preventive: it runs after the
# clone, so it detects a silent fallback to a full fetch rather than
# preventing one. Blobless is tens of MB; a full fetch is ~2.09 GB.
GIT_MB=$(du -sm .git | cut -f1)
echo "blobless clone of bundles@master: ${GIT_MB} MB" >> "$GITHUB_STEP_SUMMARY"
if [ "$GIT_MB" -gt 500 ]; then
echo "::error::blobless clone came to ${GIT_MB} MB — the blob filter is not holding (expected tens of MB). Historical tarballs are being fetched; do not let this land."
exit 1
fi
# Index from the tree, replace/add the one blob, commit, push. No checkout.
# Unlike the RC archive (idempotent on PATH), GA is idempotent on CONTENT
# and overwrites in place, so update-index --add is correct for both the
# first write and a re-release.
git read-tree HEAD
BLOB_SHA=$(git hash-object -w -- "$BUNDLE_ABS")
git update-index --add --cacheinfo "100644,$BLOB_SHA,$FILE"
TREE_SHA=$(git write-tree)
COMMIT_SHA=$(git commit-tree "$TREE_SHA" -p HEAD -m "SimpleRisk $VERSION Release")
git update-ref refs/heads/master "$COMMIT_SHA"
# Non-force: a concurrent write is rejected as a non-fast-forward rather
# than clobbered, so the worst case is a job that needs re-running.
git push origin refs/heads/master:refs/heads/master
cd - >/dev/null
if [ -n "$EXISTING_SHA" ]; then
echo "action=replaced" >> "$GITHUB_OUTPUT"
else
echo "action=committed" >> "$GITHUB_OUTPUT"
fi
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.DOWNLOADS_PUBLISHER_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
- name: Upload to S3 (unconditional overwrite)
env:
VERSION: ${{ steps.ver.outputs.version }}
run: |
set -euo pipefail
aws s3 cp "simplerisk-$VERSION.tgz" \
"s3://${{ vars.DOWNLOADS_BUCKET }}/public/bundles/simplerisk-$VERSION.tgz"
- name: Step summary
env:
VERSION: ${{ steps.ver.outputs.version }}
run: |
set -euo pipefail
{
echo "## Propagated release bundle"
echo
echo "- Version: \`$VERSION\` (${{ steps.asset.outputs.size }} bytes)"
echo "- MD5: \`${{ steps.asset.outputs.md5 }}\`"
echo "- SHA256: \`${{ steps.asset.outputs.sha256 }}\`"
echo "- bundles repo: ${{ steps.bundles.outputs.action }}"
echo "- S3 public/bundles: uploaded (overwrite)"
} >> "$GITHUB_STEP_SUMMARY"