Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions .github/workflows/collect-play.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ jobs:
git diff --cached --quiet || git commit -m "chore: update Play Console stats"
- name: Push changes
if: env.HAS_SA == 'true'
uses: ad-m/github-push-action@fe38f0a751bf9149f0270cc1fe20bf9156854365
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
run: |
# Both stats workflows push to master; retry on a rejected push.
for i in 1 2 3 4 5; do
git push origin HEAD:master && exit 0
sleep $((i * 5))
git pull --rebase origin master
done
exit 1
42 changes: 38 additions & 4 deletions .github/workflows/collect-stats.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,15 @@ on:
#pull_requests:
# branches: [ master ]
schedule:
- cron: '0 */6 * * *'
# Every 10 minutes; the "Decide whether to collect" step below only collects
# every 6 hours outside a launch window. GitHub may delay scheduled runs, so
# every row carries its own timestamp.
- cron: '*/10 * * * *'

# Never let two collections race on the push.
concurrency:
group: collect-stats
cancel-in-progress: false
Comment on lines +19 to +21

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Play updates can lose pushes

The new collect-stats group covers only stats runs, not the Play Console workflow. With stats now pushing every ten minutes, a stats commit can land while Play collects its daily data. Play then pushes from its older checkout without pulling, so its push is rejected and the collected Android data never reaches the repository.

The new git pull --rebase protects only the stats side. Give both workflows the same concurrency group, or make both writers pull and retry rejected pushes.

Knowledge Base Used: Collection automation

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 07c0318 and the follow-up: both workflows push with rebase-and-retry.


jobs:
build:
Expand All @@ -21,13 +29,32 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: Decide whether to collect
id: cadence
run: |
# High-resolution window (UTC) around the v0.14.0 launch; 10-minute
# samples correlate downloads with posts (data/launch-events.csv).
# Outside it, collect only if the last sample is ~6 h old.
WINDOW_END=2026-10-27
now=$(date -u +%s)
last=$(tail -n 1 data/stats.csv | cut -d, -f1)
age=$(( now - $(date -u -d "$last" +%s) ))
if [ "$GITHUB_EVENT_NAME" != "schedule" ] || [ "$now" -lt "$(date -u -d "$WINDOW_END" +%s)" ] || [ "$age" -ge 21000 ]; then
echo "collect=true" >> "$GITHUB_OUTPUT"
else
echo "collect=false" >> "$GITHUB_OUTPUT"
echo "Outside the launch window and last sample is only ${age}s old; skipping."
fi
- name: Install uv
if: steps.cadence.outputs.collect == 'true'
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
if: steps.cadence.outputs.collect == 'true'
run: uv sync
- name: Collect and append stats
if: steps.cadence.outputs.collect == 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
Expand All @@ -42,12 +69,19 @@ jobs:
# breakdowns). Appends only assets whose count changed since last logged.
uv run python3 collect_stats.py --assets --csv >> data/stats-assets.csv
- name: Commit
if: steps.cadence.outputs.collect == 'true'
run: |
git config --local user.email "noreply@github.com"
git config --local user.name "GitHub Action"
git add data/stats.csv data/releases.csv data/stats-assets.csv
git commit -m "chore: update stats"
- name: Push changes
uses: ad-m/github-push-action@fe38f0a751bf9149f0270cc1fe20bf9156854365
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
if: steps.cadence.outputs.collect == 'true'
run: |
# Both stats workflows push to master; retry on a rejected push.
for i in 1 2 3 4 5; do
git push origin HEAD:master && exit 0
sleep $((i * 5))
git pull --rebase origin master
done
exit 1
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ The following is manually updated:
- `chrome-weekly-users.csv` - Chrome extension weekly active users
- `firefox-daily-users.csv` - Firefox extension daily active users
- `notes.csv` - Manual entries of major/interesting events
- `launch-events.csv` - Timestamped launch posts (UTC; channel, url, note), to line up with the
high-resolution download samples. During a launch window (see `WINDOW_END` in
`.github/workflows/collect-stats.yml`) stats are collected every ~10 minutes instead of every 6 hours.


## Android vitals
Expand Down
21 changes: 20 additions & 1 deletion analyze_stats.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
import matplotlib.pyplot as plt
import pandas as pd

from collect_stats import is_updater_asset


def _load_downloads():
df = pd.read_csv("data/stats.csv", index_col="timestamp", parse_dates=True)
Expand Down Expand Up @@ -50,18 +52,35 @@ def _load_android_rating():
return df


def _load_assets():
def _load_assets(include_updater: bool = False):
"""Raw per-asset download counts over time (long format).

Columns: timestamp, tag, asset, platform, downloads. The collector logs an
asset only when its count changes, so this is sparse — see _asset_series()
for the forward-filled reconstruction.

Tauri updater assets (latest.json polls, .sig, .app.tar.gz) are dropped
unless `include_updater`: they count update checks, not downloads. See
updater_checks().
"""
df = pd.read_csv("data/stats-assets.csv")
df["timestamp"] = pd.to_datetime(df["timestamp"], utc=True)
if not include_updater:
df = df[~df["asset"].map(is_updater_asset)]
return df


def updater_checks(df=None):
"""Time series of Tauri update checks (latest.json fetches) per release.

Every running aw-tauri polls releases/latest/download/latest.json, so the
growth rate of the current release's count tracks active Tauri installs.
"""
df = _load_assets(include_updater=True) if df is None else df
df = df[df["asset"] == "latest.json"]
return _asset_series(df)


def _asset_series(df=None):
"""Wide per-(tag, asset) download counts, forward-filled over time.

Expand Down
16 changes: 15 additions & 1 deletion collect_stats.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,24 @@ def downloads_by_asset() -> list[dict]:
return rows


def is_updater_asset(asset_name: str) -> bool:
"""Assets fetched by the Tauri in-app updater, not by people downloading.

`latest.json` is polled by every running aw-tauri install (via
releases/latest/download/latest.json), and the `.sig` / `.app.tar.gz` files
are what the updater downloads to apply an update. Their counts measure
update checks and applied updates, so they are kept in the per-asset series
but excluded from the headline download total.
"""
return asset_name == "latest.json" or asset_name.endswith((".sig", ".app.tar.gz"))


def downloads(verbose=False) -> int:
"""Total downloads across all release assets."""
"""Total downloads across all release assets, excluding updater traffic."""
total = 0
for r in downloads_by_asset():
if is_updater_asset(r["asset"]):
continue
if verbose:
print(f' - {r["tag"]} [{r["platform"]}] {r["asset"]}: {r["downloads"]}')
total += r["downloads"]
Expand Down
3 changes: 3 additions & 0 deletions data/launch-events.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
timestamp,channel,url,note
2026-10-06T13:38:00+00:00,github,https://github.com/ActivityWatch/activitywatch/releases/tag/v0.14.0,v0.14.0 published
2026-10-06T17:00:00+00:00,discord,,v0.14.0 announcement in #announcements (approximate time)
53 changes: 53 additions & 0 deletions test_collect_stats.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import collect_stats
from collect_stats import is_updater_asset, platform


def test_updater_assets_are_excluded_from_downloads():
assert is_updater_asset("latest.json")
assert is_updater_asset("activitywatch-tauri-0.14.0-darwin-aarch64.app.tar.gz")
assert is_updater_asset("activitywatch-tauri-0.14.0-linux-aarch64.AppImage.sig")
assert not is_updater_asset("activitywatch-v0.14.0-windows-x86_64-setup.exe")
assert not is_updater_asset("activitywatch-tauri-0.14.0-linux-x86_64.AppImage")
assert not is_updater_asset("activitywatch-v0.14.0-macos-arm64.dmg")
Comment on lines +5 to +11

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Tests never check download totals

test_updater_assets_are_excluded_from_downloads only checks is_updater_asset(). It would still pass if the skip in downloads() were removed and updater counts returned to the headline total.

Add a test that replaces downloads_by_asset() with mixed updater and installer rows, then checks the exact total from downloads().

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 07c0318: test_downloads_total_excludes_updater_assets checks the total (320).



def test_platform():
assert platform("activitywatch-tauri-0.14.0-darwin-aarch64.app.tar.gz") == "macos"
assert platform("latest.json") == "unknown"


def test_downloads_total_excludes_updater_assets(monkeypatch):
assets = [
{
"tag": "v0.14.0",
"asset": "activitywatch-v0.14.0-windows-x86_64-setup.exe",
"platform": "windows",
"downloads": 300,
},
{
"tag": "v0.14.0",
"asset": "activitywatch-tauri-0.14.0-linux-x86_64.AppImage",
"platform": "linux",
"downloads": 20,
},
{
"tag": "v0.14.0",
"asset": "latest.json",
"platform": "unknown",
"downloads": 51,
},
{
"tag": "v0.14.0",
"asset": "activitywatch-tauri-0.14.0-darwin-aarch64.app.tar.gz",
"platform": "macos",
"downloads": 6,
},
{
"tag": "v0.14.0",
"asset": "activitywatch-tauri-0.14.0-darwin-aarch64.app.tar.gz.sig",
"platform": "macos",
"downloads": 3,
},
]
monkeypatch.setattr(collect_stats, "downloads_by_asset", lambda: assets)
assert collect_stats.downloads() == 320
Loading