Skip to content

fix(consent): load Ketch first and hold Heap and GA4 until consent - #143

Open
dg-coreylweathers wants to merge 2 commits into
mainfrom
fix/consent-ketch
Open

dg-coreylweathers wants to merge 2 commits into
mainfrom
fix/consent-ketch

Conversation

@dg-coreylweathers

@dg-coreylweathers dg-coreylweathers commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What

cli.deepgram.com and cli.staging.deepgram.com load Heap and GA4 (G-TYPC1TBCKT) before any consent, including for visitors in GDPR regions and when Ketch is blocked, and have no Ketch and no privacy control. This brings the page in line with developers.deepgram.com:

  • web/src/consent/ketch.js: docs' fern/assets/ketch.js, copied verbatim (the Ketch loader for deepgram_marketing_site plus the dgOnConsent / dgOnRevoke helpers), inlined as the first script in <head>.
  • web/public/consent/dg-consent.js: docs' built consent banner, unchanged. It mounts the footer "Your Privacy Choices" control into <div id="fern-footer"> (the id the bundle looks for), and shows a Privacy Policy link instead when Ketch is unavailable. Source commit and hashes are in web/src/consent/SOURCE.json.
  • web/src/consent/trackers.js: Heap (docs' snippet, which also fixes the old heap[o]= stub bug) and GA4 start only after Ketch grants targeted_advertising, and stop if it's withdrawn (Heap stopTracking, GA ga-disable-<id>, then the banner's reload on withdrawal). Neither starts if Ketch never loads.
  • <html class="dark"> makes the banner use its dark theme to match the site. The page has no dark: utilities, so nothing else changes.

GA4 purpose: targeted_advertising. The live Ketch config (production/gdpr/en/config.json) lists _ga and _ga_*, along with Heap's _hp2_*, under targeted_advertising. Nothing related is listed under analytics.

Checked

Built with npm ci && npm run build (node:22, Docker), served dist/, then ran Playwright with a fresh profile each time and Ketch's location stubbed:

Before (live) After (local build)
Ketch first script in <head> no Ketch yes
GB, no choice Heap, GA4, doubleclick, _ga cookies nothing loads, no cookies, banner auto-opens: Reject All · Customize · Accept All
GB Reject All / Accept All n/a nothing loads / Heap and GA4 load
US-CA (opt-out state) everything no auto-open; Heap and GA4 load (granted by default, as on docs)
Ketch blocked everything nothing loads, footer shows Privacy Policy

Notes

  • Production deploys only from a GitHub release (web-production.yml), so merging this updates staging only. cli.deepgram.com keeps the ungated trackers until the next release.
  • deepgram/deepgram-docs#1302 plans to retire cli.deepgram.com. If the redirect ships first, this goes away with it.
  • To update these files, copy them again from the same docs commit. Don't edit them here.

Release plan

Merging to main only deploys staging. Production (web-production) runs on a root v* release, and prod still loads Heap and gtag without Ketch today. Plan: request a patch release right after merge, unless docs#1302 retires cli.deepgram.com first.

Tests

web/tests/consent-gate.test.cjs (run by the new web-ci.yml on PRs touching web/**) covers: nothing loads before consent, after rejection, or with Ketch unavailable; Heap and GA4 load only on targeted_advertising; withdrawal disables GA4 and calls Heap stopTracking.

Known gap

_ga / _ga_* cookies persist after withdrawal until they expire. GA is disabled but the cookies are not deleted (same as developers.deepgram.com).

Maintenance

web/src/consent/SOURCE.json pins the vendored Ketch files by sha256, and the test checks it. If cli rebrands, re-copy both files from the same docs commit and update SOURCE.json, or the hash test will fail.

@GregHolmes GregHolmes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes

[S1] Consent gating has no checked-in behavioral regression test. The PR records manual Playwright results, but web/package.json has no test command and the repository has no test for Ketch, Heap, GA4, or the footer privacy control. A future vendor-script or tracker change can restore pre-consent requests without CI detecting it.

Add a browser test that stubs Ketch and verifies:

  • Heap and GA4 do not load before consent, after rejection, or when Ketch is unavailable
  • both load only after targeted_advertising is granted
  • withdrawal disables GA and calls Heap stopTracking

Verified: the vendored Ketch and consent-banner files match the pinned deepgram-docs commit byte-for-byte; the live Ketch config classifies both trackers under targeted_advertising; and the full-repository Node 22 build and current CI checks pass.

Operational decision: merging updates staging only. web-production.yml deploys only from a root CLI release, so decide whether this privacy fix needs an expedited production release.

@dg-coreylweathers

Copy link
Copy Markdown
Contributor Author

Thanks Greg, S1 is in.

  • web/tests/consent-gate.test.cjs runs the real ketch.js + trackers.js in node:vm against a fake Ketch, same approach as deepgram-docs#1258. Cases: before consent, after rejection, Ketch unavailable, consent helpers missing, analytics alone, grant targeted_advertising loads Heap + GA4 without a reload, withdrawal disables GA4 and calls Heap stopTracking. It also checks the vendored files against the SOURCE.json hashes.
  • npm test in web/, plus a new web-ci.yml that runs npm ci, npm test, npm run build on PRs touching web/** or pyproject.toml. Read-only permissions, actionlint clean.
  • The no-load cases also fire pending timers, intervals and load events and flush promises, so timer-, promise- and load-event-delayed fail-opens are caught. Mutation check: GA/Heap ungated, purpose swapped to analytics, stopTracking / ga-disable / dgOnRevoke dropped, Ketch defaulting to granted, loading on the queue stub, a 0 ms or 3000 ms timer fail-open, and a missing-dgOnConsent fail-open each fail the suite; restored, 8/8.
  • Manual Playwright: no tracker requests before consent, after reject, or with Ketch blocked; UK geo banner auto-opens; grant loads both without a reload.

On prod: merging only updates staging. Prod still loads Heap and gtag without Ketch today, and web-production only runs on a root v* release, so I'll ask for a patch release right after merge, unless docs#1302 retires cli.deepgram.com first.

Known gap: _ga cookies persist after withdrawal until they expire (same as docs). We disable GA but don't delete them.

@dg-coreylweathers

Copy link
Copy Markdown
Contributor Author

Pushed two more fixes from a second look at this.

  • Fonts: index.astro was preconnecting to and loading Google Fonts on every view, so the visitor's IP went to Google before consent. Inter (variable) and Fira Code are self-hosted now through @fontsource (5.3.0, months old), the preconnect and stylesheet links are gone, and a build-output test fails if a Google font host ever comes back.
  • Consent Mode: trackers.js now queues gtag('consent','default', all denied) before gtag.js and config, sends update granted when targeted_advertising is granted, and update denied on withdrawal alongside ga-disable. The test asserts the dataLayer order, including what's already in it when the gtag.js tag is appended. Removing the default fails two tests.

Checked in Chromium, Firefox and WebKit: zero heap/google requests before grant (blocked Ketch, rejected, and pre-grant), font requests only to the same origin, Inter renders, and grant loads Heap + gtag without a reload.

One thing I'm not fixing here: the footer fallback in the vendored dg-consent.js can miss Ketch's error event when the deferred script runs after it already fired, so with Ketch blocked the privacy link shows up about 15 s late (mostly Firefox). That's upstream in docs, so I'll get it fixed there and re-vendor (plus a SOURCE.json update) after.

Load the vendored Ketch loader as the first script on cli.deepgram.com and
gate Heap and GA4 on the targeted_advertising purpose. Nothing loads before
consent, after rejection, or when Ketch is unavailable; granting loads both
without a reload, and withdrawal disables GA4 and calls Heap stopTracking.
Google Consent Mode defaults are queued as denied before gtag.js, updated to
granted on consent and back to denied on withdrawal.

web/tests/consent-gate.test.cjs runs the shipped ketch.js and trackers.js in
node:vm against a fake Ketch, settling timers, intervals, promises and load
events so delayed fail-opens are caught, and checks the vendored files
against SOURCE.json. web-ci.yml runs npm ci, npm test and the build on PRs
that touch web/.
The page preconnected to and loaded Google Fonts on every view, sending the
visitor's IP to Google before consent. Inter (variable) and Fira Code are now
served from @fontsource 5.3.0, the Google Fonts links are gone, and
web/tests/build-output.test.cjs fails if a Google font host comes back.
@dg-coreylweathers

Copy link
Copy Markdown
Contributor Author

Squashed the history down to two commits, same code as before:

  • 6311875 fix(consent): Ketch first, Heap and GA4 held until consent, Consent Mode defaults, plus the consent tests and web-ci.yml.
  • 5cd0725 fix(web): self-hosted Inter and Fira Code.

Each one builds and passes npm test on its own.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants