Repository navigation
chore: bump TechEngine submodule to 45090a0 - #390
Merged
Merged
Conversation
This brings in GetTechAPI/TechEngine#133, which fixes what a review of the previous pin found. A laptop-table row is also checked under its -mobile slug, and curated records are matched by their name as well as their file name, so a drafts run no longer writes a second Quadro M5500, 3D Rage II, Radeon X800 or X850 XT PE, NVS 100 or 200, or iPhone SE. A cell with two TDPs, a base clock given for both the minimum and the maximum TDP, SGRAM or EDO memory, and ISA, VLB or several buses give no value. A CPU or GPU row without a release date is skipped instead of being filed under unknown/. GPU names leave the brand to manufacturer. The GPU backfill compares sizes of 1 MB and below again. collect.yml's three `uses:` move to the same commit. Only app/ingest, app/verify/wikipedia_gpu_backfill.py, their tests and the docs changed; the workflows, the validator and the dump code didn't. Refs #297
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Moves the TechEngine pin from
0c5012bto45090a0, and movescollect.yml's threeuses:to the same commit.The new pin brings in GetTechAPI/TechEngine#133. It fixes the problems a review of
0c5012bfound. Most of them showed up only in runs that include drafts:-mobileslug, and curated records are matched by their name as well as their file name. A drafts run no longer writes a second Quadro M5500, 3D Rage II, Radeon X800 or X850 XT PE (curated as XT Platinum), NVS 100 or 200 (curated as Quadro4 100 or 200 NVS), or iPhone SE (1st, 2nd and 3rd generation).unknown/, and ATI cards underamd/with an "AMD" name.manufacturer: "GeForce GTX 1660 Ti Mobile", not "NVIDIA GeForce GTX 1660 Ti Mobile". feat(data/gpu): ingest new SKUs #389 had to fix that by hand.Why now
With
0c5012b, a plain run on develop's data no longer writes anything. A drafts run, though, wrote duplicates of curated records and values the source cell doesn't state, so collect couldn't safely run with drafts. On the same cached pages and the same copy of the data, the two pins write:0c5012b45090a0Each duplicate the review found is now matched to its curated record, and each value it flagged is withheld. None of the 382 rows
45090a0writes lacks a year, and no GPU name starts with a brand.Checks
0c5012bis the first parent of45090a0, and the only change between them is that PR. It touches 14 files:app/ingest,app/verify/wikipedia_gpu_backfill.py, their tests anddocs/DATA_PIPELINE.md.ingest.yml,refresh.yml,verify-promote.yml,app/validate.pyand the dump code are unchanged.The ingest and backfill tests pass on
45090a0: 360 tests.Dry run of the merged collectors on a copy of develop's data (
ba13330), with every Wikipedia page fetched once:geforce-gtx-1660-ti-mobile, and45090a0names it without "NVIDIA".45090a0matches more rows to their curated records: 1,511 GPU rows instead of 1,494, and 39 phone rows instead of 35.On an empty data root it still writes 469 CPU and 363 GPU rows. Every CPU value is the same as with
0c5012b. GPU rows differ only in the dropped brand and in 33 versioned architectures.GPU backfill on a copy of the data: the 529 records it checks get the same decisions as before (1 confirm, 53 ambiguous, 475 not found), and its gate gives the same answer for all 2,075 GPU records.
pr-checks runs the full engine validator and a smoke dump, because the pin moves.
Closes #297