Repository navigation
release: CI rebuild, copyright fixes, Google Play device catalog, duplicate merges (2026-10-31) - #367
Merged
Merged
Conversation
Wikipedia lists cores only for these 23 SKUs. Each thread count is the Total Threads figure on that SKU's Intel specification page.
Adds the 23 new CPU pages. Other pages stay byte-for-byte the same; these SKUs have no benchmark scores, so CPU percentiles do not move.
Ran the existing Wikipedia image backfill tool against every smartphone record that cites a Wikipedia article and still lacks image_url. Of 432 eligible records, only 7 had a freely licensed, model-matching Commons photo in their article infobox; the rest were rejected as logo/promo images, group shots, non-free licenses, or had no infobox image at all. Closes #296
Auto-promotions from the verification layer (green+live-source or crossref-confirm). Each flip is verified:false->true only; see data/_verify/ledger.jsonl. Refs #1
The strict integrity gate (integrity_check.py --strict) has been hard-blocking weekly-refresh for at least the last 5 runs: 18 Intel Atom CPUs were each imported twice under two slugs, once without an "intel-" prefix (Wikipedia- sourced) and once with it (a later Kaggle-dataset import). These were not harmless duplicates - the two sides disagreed on release_date, threads and other fields for most pairs. Cross-checked every pair against https://en.wikipedia.org/wiki/List_of_Intel_Atom_processors: - release_date: the "atom-X" (Wikipedia) record matches the table almost exactly, often to the day. The "intel-atom-X" record's dates are frequently wrong, in two cases (Z3530, Z3560) off by a full 8 years. - threads: Wikipedia lists Hyper-Threading support for 17 of these 18 SKUs (only D2500 does not). The "intel-atom-X" record already encoded threads = 2x cores; the "atom-X" record wrongly had threads = cores for 16 of them. - clock/tdp/cache/socket/process_node: spot-checked the same way; kept whichever side matched Wikipedia (mostly the "atom-X" side, e.g. Z3560's "intel-atom-z3560" record had clock_ghz=1.0 vs Wikipedia's stated 1.83). Each merged record keeps the union of both sides' source_urls and survives under the shorter "atom-X" slug/path. `integrity_check.py --strict` now reports 0 hard anomalies. Closes #296
Correct release dates, one clock, and two TDPs where the Wikipedia rows for that exact record agree, and move changed-year files to the matching directory.
The Northwood-128 row lists 52.8 W. The CPU field is an integer, and 53 W is that value already stored.
image_url and logo_url now point only at Wikimedia Commons files. State that those files are not covered by the dataset's CC BY-SA 4.0 grant and how to credit them. Refs #296
data/_verify/cache/wikipedia_html/ held 325 full Wikipedia article pages (61 MB), several unrelated to any device. TechEngine's wikipedia_smartphone_backfill writes this page cache during apply runs, and it was swept into 0bb442a by accident. Untrack it and ignore the cache directory so it stays local. Refs #296
The Commons file Imgi-17-samsung-galaxy-tab-s11-ultra-galaxy-ai-scaled.webp names its source as "Samsung Website" and its author as Samsung; the CC0 tag is the uploader's, not Samsung's grant. Remove the three image fields the backfill inserted and record a bad_license decision in its cache so a later run does not put the file back. Refs #296
Mirror of the engine rule (TechEngine ADR-017): reject any image_url outside upload.wikimedia.org/wikipedia/commons at any depth, require image_license and image_attribution next to a photo, and accept only Commons files for logo_url. The data in this PR already satisfies it. Refs #296
… page Device cards rendered any image_url, which put hotlinked third-party product photos on the public site. Render only upload.wikimedia.org/wikipedia/commons files, and add a 'Photo: Wikimedia Commons' credit linking to the file page, which lists the author and license (CC BY-SA allows attribution by link). Refs #19
Smartphone pages already carry image_attribution and image_license, so the credit reads 'Photo: <author>, <license> · Wikimedia Commons' and still links to the file page; records without those fields keep the plain Commons credit. Refs #19
Member
Author
🔎 Data verification — Tier 0 (offline existence/trust)Scored by 13/13 categories ? 213,649 records ? diff base main@d3aac35 Changed records in this PR76603 record(s) assessed. %%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
title Verification bands — all records
"Green" : 14325
"Yellow" : 62224
"Red" : 54
Full-dataset baselineAs of 2026-10-06T12:12:50Z (daily
green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to |
Member
Author
TechEngine change review: PASS13/13 categories ? 213,649 records ? diff base main@d3aac35
Changed data
Changed record examplesbrand added
brand modified
soc modified
smartphone added
smartphone modified
smartphone deleted
tablet modified
tablet deleted
watch modified
watch deleted
pda modified
pda deleted
gpu added
gpu modified
cpu added
cpu modified
cpu deleted
laptop modified
laptop deleted
monitor modified
monitor deleted
website modified
device_catalog added
Heuristic review
Changed site
Modified site files
|
Member
Author
TechEngine validation stats: PASS13/13 categories ? 213,649 records ? diff base main@d3aac35
Data summary
Warning Tracked verified coverage is below 50% for device_catalog 0.0% (0/20366), software 0.8% (358/42493), brand 4.1% (123/3022), laptop 6.6% (129/1966), tablet 8.9% (293/3305), monitor 9.1% (21/231), soc 13.4% (284/2117), all 18.3% (39048/213649), and 3 more. Validation notes
Key output:
Homepage build: |
Stop all automation in this repository. Several workflows used the ENGINE_TOKEN and PROJECT_TOKEN personal access tokens to relay events to TechEngine and add PRs to the triage project, so those calls ran as a real account rather than as GitHub Actions. This removes bump-engine, deploy-pages, dump-develop, notify-engine, pr-metadata, request-engine-pr-validation, validate-data, verify-command and verify-report. All nine are also disabled in the repository's Actions settings, so the copies still on main stay inert until the next release removes them there too. PR validation, the dump refresh and the Pages deploy no longer run automatically. Refs #297
Seungpyo1007
added
enhancement
Bring back the Pages deploy without any token and without committing generated files. pages.yml runs on a push to main (a merged release PR) or by hand. It validates the data, regenerates site/public/v1, openapi.json and data/_verify/status.json with the pinned TechEngine, checks every collection's counts against data/, builds the site and publishes it. A manual run on any other branch is a dry run. The build job can only read the repository; only the deploy job can write, and only to Pages. Engine steps run with every proxy pointed at a closed port, so they cannot reach third-party sites. Actions are pinned by SHA. Regenerating at deploy time also fixes what the committed dump gets wrong today: the software pages named build and env, which .gitignore keeps out of git, and records changed since the last dump refresh. Refs #297
… check pr-checks.yml runs on every PR into develop or main with a read-only token and no secrets: the bundled validator, TechEngine's validator on the changed records (every record when the pin moves), the integrity gate, a Tier 0 report and, when site/ changes, the site build. Results go to the job summary and file annotations; nothing is commented. final-check.yml runs when a maintainer adds the final-check label to a PR. It builds what pages.yml would publish from the PR, and TechEngineBot posts one comment with the verdict, edited in place when the label is added again. Only that comment uses TECHENGINEBOT_TOKEN, and the job refuses anything but a fine-grained token of TechEngineBot. Fork PRs get no comment. Dependabot opens the TechEngine submodule bump weekly and grouped action updates monthly, at most one open PR each, into develop. Nothing fills the PR sidebar automatically any more, so the template asks for it. Refs #297
pages.yml now generates site/public/v1 and openapi.json at deploy time from the data it publishes, so the committed copy only went stale and grew the repository with every refresh. Remove it from the index (the files stay on disk) and ignore it, keeping only the hand-made v1/games/index.json pointer. Anchor the build/, dist/, env/, venv/, .venv/ and htmlcov/ rules to the repository root: unanchored, build/ and env/ also matched the software pages v1/software/build and v1/software/env, so those two never reached the site. The growth chart read its points from the history of v1/index.json. build-history.mjs keeps those and continues from the first-parent commits that changed data/, counting one JSON file per record as the manifest does. Refs #297
The job summary is not readable through the API, so a passing run left no record of what it checked anywhere but the web page. Refs #297
The first full run on GitHub took 49 minutes to generate the API, which left pages.yml (full-history checkout, upload) too little room under 90. Refs #297
Seungpyo1007
changed the title
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request. Check if the Files changed in this pull request are included in default exclusions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 2026-10-31
Everything merged into
developsince the 2026-09-27 release (#315).Copyright fixes
data/LICENSE.md, and validates thatimage_url/logo_urlonly link Wikimedia Commons files.Google Play identity data
device_catalogentries and the device makers they reference (ADR-016).Accuracy
New records and citations
CI
GITHUB_TOKEN:pages.ymlgenerates the API frommainat deploy time and publishes the site.pr-checks.ymlchecks every PR read-only.final-check.ymllets a maintainer ask TechEngineBot for one final-check comment.The generated API is no longer committed (#377). Merging this runs
pages.ymlonmain, which regeneratessite/public/v1from this data. That also brings the photo removals from #365 and thesoftware/buildandsoftware/envpages to the live API.Closes #1