Skip to content

chore: bump TechEngine submodule to 45090a0 - #390

Merged
Seungpyo1007 merged 1 commit into
developfrom
chore/techengine-45090a0
Nov 2, 2026
Merged

Seungpyo1007 merged 1 commit into
developfrom
chore/techengine-45090a0

Conversation

@Seungpyo1007

Copy link
Copy Markdown
Member

What

Moves the TechEngine pin from 0c5012b to 45090a0, and moves collect.yml's three uses: to the same commit.

The new pin brings in GetTechAPI/TechEngine#133. It fixes the problems a review of 0c5012b found. Most of them showed up only in runs that include drafts:

  • Duplicates. A laptop-table row is also checked under its -mobile slug, and curated records are matched by their name as well as their file name. A drafts run no longer writes a second Quadro M5500, 3D Rage II, Radeon X800 or X850 XT PE (curated as XT Platinum), NVS 100 or 200 (curated as Quadro4 100 or 200 NVS), or iPhone SE (1st, 2nd and 3rd generation).
  • Values. These cells now give no value:
    • a TDP cell with two numbers ("92.6 68 W", two Opteron parts in one row)
    • a base clock given for both the minimum and the maximum TDP. The Core i9-11980HK used to get its 65 W base, 3.3 GHz.
    • SGRAM, EDO, VRAM, WRAM or FPM memory next to SDR
    • ISA, EISA, VLB or MCA buses, and "AGP 4x,8x"
  • No release date. A CPU or GPU row without a date is skipped. It used to be filed under unknown/, and ATI cards under amd/ with an "AMD" name.
  • Names. GPU names leave the brand to manufacturer: "GeForce GTX 1660 Ti Mobile", not "NVIDIA GeForce GTX 1660 Ti Mobile". feat(data/gpu): ingest new SKUs #389 had to fix that by hand.
  • Architecture. GT2xx, GF11x and GK2xx chips are Tesla 2.0, Fermi 2.0 and Kepler 2.0, as the curated records name them.
  • GPU backfill. Memory sizes of 1 MB and below are compared again. A record whose memory Wikipedia contradicts stays ambiguous instead of being confirmed.

Why now

With 0c5012b, a plain run on develop's data no longer writes anything. A drafts run, though, wrote duplicates of curated records and values the source cell doesn't state, so collect couldn't safely run with drafts. On the same cached pages and the same copy of the data, the two pins write:

Drafts run 0c5012b 45090a0
CPU 262 189
GPU 278 191
Smartphone 5 2

Each duplicate the review found is now matched to its curated record, and each value it flagged is withheld. None of the 382 rows 45090a0 writes lacks a year, and no GPU name starts with a brand.

Checks

  • 0c5012b is the first parent of 45090a0, and the only change between them is that PR. It touches 14 files: app/ingest, app/verify/wikipedia_gpu_backfill.py, their tests and docs/DATA_PIPELINE.md. ingest.yml, refresh.yml, verify-promote.yml, app/validate.py and the dump code are unchanged.

  • The ingest and backfill tests pass on 45090a0: 360 tests.

  • Dry run of the merged collectors on a copy of develop's data (ba13330), with every Wikipedia page fetched once:

    Category Written Already curated Incomplete or uncertain
    gpu 0 1,512 293
    cpu 0 1,429 394
    smartphone 0 39 2
    • On the data from before feat(data/gpu): ingest new SKUs #389, both pins write only geforce-gtx-1660-ti-mobile, and 45090a0 names it without "NVIDIA".
    • 45090a0 matches more rows to their curated records: 1,511 GPU rows instead of 1,494, and 39 phone rows instead of 35.
  • On an empty data root it still writes 469 CPU and 363 GPU rows. Every CPU value is the same as with 0c5012b. GPU rows differ only in the dropped brand and in 33 versioned architectures.

  • GPU backfill on a copy of the data: the 529 records it checks get the same decisions as before (1 confirm, 53 ambiguous, 475 not found), and its gate gives the same answer for all 2,075 GPU records.

  • pr-checks runs the full engine validator and a smoke dump, because the pin moves.

Closes #297

This brings in GetTechAPI/TechEngine#133, which fixes what a review of
the previous pin found. A laptop-table row is also checked under its
-mobile slug, and curated records are matched by their name as well as
their file name, so a drafts run no longer writes a second Quadro
M5500, 3D Rage II, Radeon X800 or X850 XT PE, NVS 100 or 200, or
iPhone SE. A cell with two TDPs, a base clock given for both the
minimum and the maximum TDP, SGRAM or EDO memory, and ISA, VLB or
several buses give no value. A CPU or GPU row without a release date
is skipped instead of being filed under unknown/. GPU names leave the
brand to manufacturer. The GPU backfill compares sizes of 1 MB and
below again.

collect.yml's three `uses:` move to the same commit. Only app/ingest,
app/verify/wikipedia_gpu_backfill.py, their tests and the docs
changed; the workflows, the validator and the dump code didn't.

Refs #297
@Seungpyo1007 Seungpyo1007 added bug Something isn't working ci CI and workflow changes labels Nov 2, 2026
@Seungpyo1007
Seungpyo1007 merged commit d7f4583 into develop Nov 2, 2026
5 checks passed
@Seungpyo1007
Seungpyo1007 deleted the chore/techengine-45090a0 branch November 2, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci CI and workflow changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Engine, TechEngineBot and automation (workstream)

2 participants