Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/traffic-stats.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,13 @@ jobs:
run: npm ci
working-directory: tools/catalog-build

# GitHub repository traffic (views, clones, referrers) is paused: it needs a token that
# expires every few days under organization policy, and the site's page analytics come
# from Clarity. Set the repository variable COLLECT_GITHUB_TRAFFIC to "true" and replace
# the TRAFFIC_TOKEN secret to resume.
- name: Collect and persist GitHub traffic data
id: github-traffic
if: ${{ vars.COLLECT_GITHUB_TRAFFIC == 'true' }}
continue-on-error: true
env:
GH_TOKEN: ${{ secrets.TRAFFIC_TOKEN }}
Expand Down Expand Up @@ -272,6 +277,10 @@ jobs:

echo "✅ Saved $DATA_DIR/$DATE.json"

- name: Note that GitHub traffic collection is paused
if: ${{ vars.COLLECT_GITHUB_TRAFFIC != 'true' }}
run: echo "::notice::GitHub repository traffic collection is paused. Page analytics come from Clarity. To resume, set the repository variable COLLECT_GITHUB_TRAFFIC to true and replace the TRAFFIC_TOKEN secret."

- name: Build resource stats
id: stats
if: always()
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This file captures only selected high-level milestones.
* Added a shareable page for every resource; archived resources are now hidden by default; added an Agent 365 highlight row (driven by the `agent-365` tag) and "New" badges to the landing page.
* Cataloged 23 previously unlisted scripts, snippets, and samples (Viva Engage administration, Teams Phone and Rooms, SimpleGraph, ESS extensions) and archived scripts that depend on retired modules or services.
* Archived the prompt samples and removed a 41 MB archived demo video.
* Added issue forms, a refreshed pull request template, and a manual traffic collection command.
* The daily **Collect Traffic Data** workflow no longer collects GitHub repository traffic (views, clones, referrers), which needs a token that expires every few days under organization policy. It skips that step with a notice and keeps collecting Clarity page analytics and discussion votes. Set the repository variable `COLLECT_GITHUB_TRAFFIC` to `true` and replace `TRAFFIC_TOKEN` to resume.

### April 20, 2018

Expand Down
23 changes: 16 additions & 7 deletions tools/catalog-build/test-traffic-workflow.js
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@ assert(hasUnsafeConflict(['resource-stats.json', 'traffic-data/2026-09-14.json']
'A mixed conflict set must stop synchronization instead of overwriting source data.');
assert(githubTraffic?.['continue-on-error'] === true,
'GitHub traffic failure must not block the independent Clarity source.');
assert(githubTraffic?.if?.includes("vars.COLLECT_GITHUB_TRAFFIC == 'true'"),
'GitHub traffic collection must run only when the COLLECT_GITHUB_TRAFFIC repository variable is true.');
const pauseNote = collectSteps.find(step => step.name === 'Note that GitHub traffic collection is paused');
assert(pauseNote?.if?.includes("vars.COLLECT_GITHUB_TRAFFIC != 'true'") && pauseNote.run.includes('::notice::') && !pauseNote.run.includes('::error::'),
'A paused GitHub traffic collection must say so as a notice, never as an error.');
assert(stats?.['continue-on-error'] === true && stats.run.includes('--require-clarity'),
'Clarity must report unavailability without blocking a healthy GitHub snapshot.');
assert(stats?.if === 'always()', 'Stats must run even when GitHub traffic collection fails.');
Expand All @@ -87,8 +92,8 @@ assert(persist.run.includes('if [ "${{ steps.stats.outputs.generated }}" = "true
function evaluateCondition(expression, state) {
const replacements = new Map([
['always()', true],
["steps.github-traffic.outcome == 'success'", state.github],
["steps.github-traffic.outcome == 'failure'", !state.github],
["steps.github-traffic.outcome == 'success'", state.github === 'success'],
["steps.github-traffic.outcome == 'failure'", state.github === 'failure'],
["steps.stats.outputs.clarity_available == 'true'", state.clarity],
["steps.stats.outcome == 'failure'", !state.clarity],
["steps.persist.outcome == 'failure'", state.persistFailed]
Expand All @@ -101,11 +106,15 @@ function evaluateCondition(expression, state) {
}

const combinations = [
{ github: true, clarity: true, persistFailed: false, persist: true, fail: false },
{ github: true, clarity: false, persistFailed: false, persist: true, fail: true },
{ github: false, clarity: true, persistFailed: false, persist: true, fail: true },
{ github: false, clarity: false, persistFailed: false, persist: false, fail: true },
{ github: true, clarity: true, persistFailed: true, persist: true, fail: true }
{ github: 'success', clarity: true, persistFailed: false, persist: true, fail: false },
{ github: 'success', clarity: false, persistFailed: false, persist: true, fail: true },
{ github: 'failure', clarity: true, persistFailed: false, persist: true, fail: true },
{ github: 'failure', clarity: false, persistFailed: false, persist: false, fail: true },
{ github: 'success', clarity: true, persistFailed: true, persist: true, fail: true },
// Paused GitHub traffic: Clarity alone keeps the run green and persisted.
{ github: 'skipped', clarity: true, persistFailed: false, persist: true, fail: false },
{ github: 'skipped', clarity: false, persistFailed: false, persist: false, fail: true },
{ github: 'skipped', clarity: true, persistFailed: true, persist: true, fail: true }
];

for (const combination of combinations) {
Expand Down
9 changes: 8 additions & 1 deletion traffic-data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ Daily collection is committed to the durable `automation/traffic-stats-staging`
branch so Clarity's three-day export window is preserved without daily pull requests.

**Files:**
- `YYYY-MM-DD.json` — Full daily snapshot (views, clones, referrers, popular paths)
- `YYYY-MM-DD.json` — Full daily snapshot (views, clones, referrers, popular paths). Collection of these is paused; see below.
- `clarity-views.json` — Accumulated Clarity page views per resource, the source of the view counts on the site.

Data is collected daily at 06:00 UTC via GitHub Actions. Each Friday at 08:00
`America/Chicago` (including daylight saving time), the workflow updates one
Expand All @@ -15,6 +16,12 @@ open weekly review. If the optional `AUTOMATION_PAT` is not configured or cannot
create pull requests, the workflow prints a manual compare URL; daily collection
continues safely on the staging branch.

## GitHub repository traffic is paused

Since 2026-10-29 the workflow no longer collects GitHub's repository traffic (views, clones, referrers, popular paths, stars and forks). That API needs a token with Administration read access, and organization policy limits fine-grained tokens to a few days, so the daily run failed and the alert issue stayed open. Page analytics for the catalog come from Clarity, which the workflow still collects every day, along with discussion votes. The daily run skips the GitHub step with a notice and stays green.

To resume, create a token that meets the organization's policy, save it as the `TRAFFIC_TOKEN` secret, and set the repository variable `COLLECT_GITHUB_TRAFFIC` to `true` (Settings, Secrets and variables, Actions, Variables). The next run collects again. GitHub keeps only 14 days, so anything not collected while paused is gone.

## Manual collection

When the scheduled workflow cannot authenticate (for example, its `TRAFFIC_TOKEN` expired), collect a snapshot by hand. GitHub keeps only a rolling 14 days of traffic, so run this at least every two weeks to avoid gaps. Each snapshot overlaps the previous one, and analysis unions days by date, so extra runs never double count.
Expand Down
Loading