Refresh only cache objects near expiry

[buildpulser]

The cost

The commit post-step keeps an active tenant's referenced layers alive by copying them in place — a metadata REPLACE, no data transfer — which resets the bucket's LastModified expiry clock. It did that for every referenced object on every build.

Each copy is billed as a Tier1 PUT. Those refreshes measured ~3.8M Tier1 requests over 20 days — 83% of the entire S3 bill for runner caching, far more than storage itself:

line	20 days	share
Requests Tier1	$19.11	83%
Storage	$2.92	13%
Requests Tier2	$0.77	3%
Transfer	$0.24	1%

Touching an object with six days left on a seven-day rule buys nothing.

The change

One LIST per prefix (Tier2, ~12x cheaper per call, paginated) gives every object's age; only those near expiry are copied.

const snapAges = objectAgesDays('snap-ca');
const snapInc = snapKeys
  .filter(k => nearExpiry(snapAges, k, REFRESH_WHEN_AGE_DAYS))
  .flatMap(k => ['--include', k]);

The GC property is unchanged — this is the part to check

Orphan collection was the point of the original design, and it still works. An object is refreshed only when this build references it and it is near expiry. Unreferenced orphans are still never touched and still age out.

Skipping a young referenced object cannot expire it: by definition it has days of life left, and any build inside that window refreshes it then.

Fails open

Both unknown cases refresh exactly as before:

• the listing failed → ages is null → refresh everything
• a key is absent from the listing (raced with an upload) → refresh it

Refreshing unnecessarily costs one request. Skipping wrongly costs the layer.

Threshold: why 1 and not 3

Worst-case object age is threshold + longest inter-build gap, and that must stay under the 7-day lifecycle.

threshold	worst-case age	quiet-stretch tolerance
0 (today)	~2.4d	~4.6d
1 (shipped)	~3.4d	~3.6d
3	~5.4d	~1.6d

An active repo builds many times an hour, so at 1 day the overwhelming majority of refreshes are already skipped. Raising it to 3 removes little more while cutting the tolerance by more than half. Overridable via BP_CACHE_REFRESH_AGE_DAYS; the reasoning is in the code so nobody raises it without redoing the arithmetic.

Tests — the repo had none

No test script, no test directory, @vercel/ncc the only devDependency. Parsing and the fail-open rule move to src/refresh-policy.js so they are testable; only the AWS call and error handling stay in post.js.

Adds node --test (no new dependencies) and 11 cases:

• aws s3 ls parsing, and that timestamps are read as UTC — a naive parse would shift ages by the TZ offset and could flip a boundary decision
• key relativisation for the nested blobs/sha256/ shape
• malformed lines skipped, not thrown on — a post-step that runs after a successful build must not crash
• threshold inclusivity at the boundary
• both fail-open paths
• the shipped threshold stays under the lifecycle with >3 days headroom
• a threshold of 5 would expire referenced layers — the check to re-run before ever raising it

Ships as v2

@v1 pins are untouched; nobody moves silently.

Test plan

• npm test — 11/11
• node --check src/post.js
• npm run build, and verified the logic is present in the minified dist/post.js by string literal (the exit code alone lies — ncc was missing and the build silently no-opped on the first attempt)
• Prove in development on a docker-builder tenant: new log line appears, refresh count drops, objects still do not expire
• Only then tag v2