Repository navigation

Add a reference lint for transforms, tables and transform tests - #40

Closed

galdre wants to merge 220 commits into

mainfrom

galdre/reference-lint

Closed

Add a reference lint for transforms, tables and transform tests#40
galdre wants to merge 220 commits into
mainfrom
galdre/reference-lint

Commits on Oct 27, 2026
- Add a reference lint for transforms, tables and transform tests
  galdre
  committed
  62a5d6b
- Some WIP typescript cleanup
  galdre
  committed
  87aff78
- More cleanup
  galdre
  committed
  e075594
- PR review feedback
  galdre
  committed
  01fa0f3
- More cleanup
  galdre
  committed
  93b1100
Commits on Oct 28, 2026
- CI: H2 parity, refusal messages, masking, Node 24
  galdre
  committed
  16f73d3
- Lint: row, expectation, collection and incremental-tag rules
  galdre
  committed
  805f624
- Read YAML as Metabase does; behavior cleanups
  galdre
  committed
  cbab908
- Parse SQL with Metabase's SQLGlot; require default schemas
  galdre
  committed
  8c5fb4d
- Lint: engine rules and Database default schemas
  galdre
  committed
  e3c3c6c
- Lint: column checks with Metabase's SQL functions
  galdre
  committed
  1ee5c6e
- fix CI
  galdre
  committed
  f1106aa
- Read null MBQL schemas as no schema
  galdre
  committed
  63199df
- Read test SQL on the query's database
  galdre
  committed
  a468c98
- Report a non-query transform with no target database
  galdre
  committed
  d602102
- Name the path and reason in schema findings
  galdre
  committed
  f2e3350
- Bound snippet pasting as Metabase does
  galdre
  committed
  6625c08
- Report a parser answer that is not JSON as unread
  galdre
  committed
  494fc2c
- Name the Metabase versions the SQL parser matches
  galdre
  committed
  ddcae48
- Read a YAML timestamp as a Date with no time
  galdre
  committed
  343984b
- Mark parser answers so Pyodide converts them intact
  galdre
  committed
  d248152
- Load the SQL parser on the first SQL read
  galdre
  committed
  55aad0f
- Forbid catch clauses where the lint's checks run
  galdre
  committed
  d90dc4e
- Compare numeric YAML keys as SnakeYAML does
  galdre
  committed
  5de224e
- Warn on null-schema references under a non-null default schema
  galdre
  committed
  f1a630b
- Label a table with no schema in finding messages
  galdre
  committed
  3f6e94e
- List the README version note among vendor refresh steps
  galdre
  committed
  5f95752
- State who controls the alpha publish job and token
  galdre
  committed
  5f0ed12
- Check the vendored SQLGlot wheel's SHA-256 against PyPI
  galdre
  committed
  ffeb414
- Refuse an alpha base version that is not X.Y.Z
  galdre
  committed
  4fa6d53
- Give the Metabase sources check a read-only token
  galdre
  committed
  1726ccf
- Pin setup-bun by commit in the publishing workflows
  galdre
  committed
  cd493d8
- Say Already published when the alpha version existed
  galdre
  committed
  1487f9f
- Correct five comments that misdescribe the code
  galdre
  committed
  b56721d
- Correct a test name, two comments and a step name
  galdre
  committed
  b53afdb
- Install an already published alpha by its version
  galdre
  committed
  4dbce71
- Snapshot lint findings only; move one transform per engine tree
  galdre
  committed
  0543cd3
- Drop lint tests that repeat snapshot entries
  galdre
  committed
  5751811
- Stop comparing mutation severities with the severity table
  galdre
  committed
  a752513
- Pin reads through cards, measures and unread sources
  galdre
  committed
  91725ee
- Derive the rule list from the severity map
  galdre
  committed
  ec743dd
- Build engine facts from one base entry; cite Metabase commit
  galdre
  committed
  d7d0a2a
- Compare test inputs with sameTable; drop inputsCollide
  galdre
  committed
  ebbfd66
- Fold repeated parse caches and small helpers
  galdre
  committed
  12da45f
- Read INTERSECT and EXCEPT field references as unread
  galdre
  committed
  24025ee
- Trust the pinned parser's answer shapes
  galdre
  committed
  a7a846e
- Exit 2 when the SQL parser cannot start
  galdre
  committed
  b40907e
- Name a self-referencing source's chain once
  galdre
  committed
  f96e8d7
- Find each test's query transform once
  galdre
  committed
  c6d6605
- Keep the test mutations out of the published package
  galdre
  committed
  e33d4aa
- Read and validate each file once; js-yaml to devDependencies
  galdre
  committed
  2c694e6
- Make pyodide an optional dependency
  galdre
  committed
  faf2f33
- Name a template tag after a schema as written
  galdre
  committed
  2830fc3
- Start the SQL parser again after a failed start
  galdre
  committed
  271b82e
- Move the lint's details from the README to docs/lint.md
  galdre
  committed
  1f1cc36
- Test the README's rule table against the lint's rules
  galdre
  committed
  a9bba5f
- Point the sources check to docs/lint.md
  galdre
  committed
  75c5527
- Drop figures that go stale from the lint docs
  galdre
  committed
  438ebfd
- Document every exit code of the lint and validator
  galdre
  committed
  94f3691
- Correct the engine notes in docs/lint.md
  galdre
  committed
  57790c9
- Say a null default_schema can mean not synced
  galdre
  committed
  d66aba9
- Report SQL whose columns the lint cannot read
  galdre
  committed
  d0993fc
- Ship docs/lint.md and link it absolutely
  galdre
  committed
  5f9c32f
- Move the publishing notes to CONTRIBUTING.md
  galdre
  committed
  9e14633
- Link the README's files on GitHub
  galdre
  committed
  30d2b48
- Name the model of a schema whose serdes/meta is a path
  galdre
  committed
  87d22f3
- Find entity files as Metabase does; model from serdes/meta
  galdre
  committed
  6a859d8
- Spec: list the folders Metabase imports
  galdre
  committed
  d34df79
- Warn on a file Metabase skips for having no serdes/meta
  galdre
  committed
  52fc488
- Pass a file of a model Metabase imports and no schema describes
  galdre
  committed
  4b43373
- Follow no symbolic link to a folder
  galdre
  committed
  b46ebf1
- Check the copied model list against Metabase in CI
  galdre
  committed
  289a7ae
- Spec: files Metabase skips, and symbolic links
  galdre
  committed
  940258d
Commits on Oct 29, 2026
- Run the example transforms on a Postgres warehouse
  galdre
  committed
  d6c4ecc
- Add a mutation that breaks four tests Metabase refuses
  galdre
  committed
  5322d2e
- Wait for a git sync import by its task id
  galdre
  committed
  90d220f
- Import workflow: Postgres warehouse, and a tree of refused tests
  galdre
  committed
  6b43d42
- Parity: locate unremapped references, print test refusal messages
  galdre
  committed
  7b7b248
- Make the lint experimental and Pyodide an optional peer
  galdre
  committed
  57cb946
- Copy Metabase's sql_tools.py at ad0af5b6b03
  galdre
  committed
  d805d3d
- Need a default schema for two schemas only within a test
  galdre
  committed
  7fb19d5
- Name the commit the engine table was copied at
  galdre
  committed
  78119d8
Commits on Oct 30, 2026
- Compare imported table names exactly and SQL names ignoring case
  galdre
  committed
  c617423
- Find the tables an expectation reads after Metabase's rewrite
  galdre
  committed
  8d15632
- Parse template tags as Metabase does, skipping SQL comments
  galdre
  committed
  8715cd1
- Read no column that is an alias or a keyword
  galdre
  committed
  f968853
- Compare read columns in the case the warehouse uses
  galdre
  committed
  b3b8bb7
- Read files and times as Metabase's import does
  galdre
  committed
  f8974a6
- Read a transform test's SQL as written
  galdre
  committed
  103d62f
- Strip large literal lists before reading tables and columns
  galdre
  committed
  034b274
- Skip null-schema-reference on a table defined with no schema
  galdre
  committed
  44e476b
- Read a blank schema by Java's whitespace
  galdre
  committed
  4e0139f
- List merged and aliased YAML keys where Metabase reads them
  galdre
  committed
  92de442
- Walk a template tag named like an Object property
  galdre
  committed
  a2dd29c
- Accept only plain objects as records
  galdre
  committed
  1fa64c3
- Refuse an async callback to withSqlParser by type
  galdre
  committed
  d194f86
- Sort the used fields and missing table aliases
  galdre
  committed
  e3638c8
- Correct docstrings that disagree with their code
  galdre
  committed
  a892e6a
- Treat a document as valid only when its schema passed
  galdre
  committed
  36b938c
- Keep the items of a checked array unknown
  galdre
  committed
  5dc0dde
- Make fieldSpecOf exhaustive over field spec types
  galdre
  committed
  ffb335f
- Tie a lint finding's severity to its rule
  galdre
  committed
  cea7b72
- Say where to install pyodide
  galdre
  committed
  c710dd9
- Name the file and cause of each missing default schema
  galdre
  committed
  f610aa2
- Exit 2 on usage errors and list exit codes once
  galdre
  committed
  a25637c
- Say when the folder is missing or is a file
  galdre
  committed
  c178f21
- Check and echo each --default-schema
  galdre
  committed
  9101393
- Warn on a transform with an empty native query
  galdre
  committed
  3dbea2c
- Give fix guidance in dangling-transform-id and transform-cycle
  galdre
  committed
  5e19461
- Say what an entity_id needs, and how to make one
  galdre
  committed
  2e24e6d
- Document the fields of a lint finding
  galdre
  committed
  279600d
- Report a file that is not valid UTF-8 as such
  galdre
  committed
  372f64d
- Show the SQL line a ParseError names
  galdre
  committed
  d84c948
- Test what a deliberate break of the lint leaves green
  galdre
  committed
  2923041
- Test that --default-schema refuses an empty database name
  galdre
  committed
  ce35027
- State the folder the lint expects, and correct docs wording
  galdre
  committed
  8f3ba8a
- Set the package version to 1.6.2
  galdre
  committed
  23b228c
- Hash and truncate the branch in alpha versions and tags
  galdre
  committed
  be7d6f2
- Fail the alpha version step on a refused base version
  galdre
  committed
  be3cd21
- Tokenize workflow run scripts in the workflow guards
  galdre
  committed
  f1d0039
- Document alpha builds in CONTRIBUTING
  galdre
  committed
  3b82318
- Type-check, lint and format the scripts folder
  galdre
  committed
  f8f6ed7
- Compare sql_tools.py byte for byte only off pull requests
  galdre
  committed
  4387921
- Copy Metabase's sql_tools.py at 283fc309bf2
  galdre
  committed
  68369c6
- Run the import workflow daily
  galdre
  committed
  cdee4be
- Compare test refusals in Metabase's check order
  galdre
  committed
  35c2847
- Pin that wait-for-import waits for its own task
  galdre
  committed
  3598de8
- Fail a parity test whose refusal cannot be located
  galdre
  committed
  6dd534e
- Resolve entity_id references by model, warn on same-model duplicates
  galdre
  committed
  b53d4ea
- Remove the unknown-table rule
  galdre
  committed
  7963bfa
- Report null-schema references as errors on engines with schemas
  galdre
  committed
  9754cf9
- Warn on a duplicate YAML key instead of refusing the file
  galdre
  committed
  24c03ee
- Check Card, Dashboard and Document files against their schemas
  galdre
  committed
  4772226
- Warn on a transform or card whose engine is unknown
  galdre
  committed
  43531b3
- Refuse a YAML document whose aliases expand past a budget
  galdre
  committed
  a7cd2fc
- Name the npx form that installs Pyodide with the lint
  galdre
  committed
  93951a6
- Find a publish behind env, time and other wrappers
  galdre
  committed
  1a09842
- Fail a file whose top-level key is not text
  galdre
  committed
  2eff64c
- Read 24:00, offsets and signed years as Metabase does
  galdre
  committed
  a9b427a
- Suggest a fresh entity id only for a file's own id
  galdre
  committed
  37b2b3a
- Test the byte order mark at the limit through a file
  galdre
  committed
  1255673
- Say in the README that a non-UTF-8 file fails
  galdre
  committed
  07f8cd4
- Refuse a parser use that may answer a promise
  galdre
  committed
  8f44093
- State the order of referencedTables' tables
  galdre
  committed
  69da4b4
- Check the shape of the SQL parser's answers
  galdre
  committed
  f76ee8a
- Say that column checks read only a single SELECT
  galdre
  committed
  3449ad9
- Report a table read by MBQL and by SQL once
  galdre
  committed
  1fdcd05
- Keep only the type check in the findings test
  galdre
  committed
  782861a
- Correct the lint docs on case, refusals and limits
  galdre
  committed
  50175cd
- Compare each exit code's meaning in the help and README
  galdre
  committed
  53efbd4
- Exit 2 on an unexpected error, not 1
  galdre
  committed
  2438b61
- Say which folders must hold YAML files
  galdre
  committed
  aaf444d
- Echo --default-schema once the lint has read the folder
  galdre
  committed
  73793b2
- Pin where Metabase checks unsupported transforms and duplicates
  galdre
  committed
  cec10b4
- Read a quoted name as a column, not a keyword
  galdre
  committed
  bb306fd
- Fail a YAML timestamp where a schema wants a map
  galdre
  committed
  cf0628d
- Name the incremental tag a table variable, as Metabase does
  galdre
  committed
  20583b5
- Pair each transform kind with its own driver feature
  galdre
  committed
  cbe2ba4
- Quote non-string row keys and say how to fix them
  galdre
  committed
  40b9547
- Fold every column name to lower case on Redshift
  galdre
  committed
  bccc210
- Compare transform targets ignoring case on Redshift
  galdre
  committed
  bc1c3ea
- Read SELECT aliases per scope and by engine
  galdre
  committed
  5b923b0
- Compare sql_tools.py on every run and script the copy
  galdre
  committed
  0244877
- Warn on each read database whose engine is unknown
  galdre
  committed
  3df541d
- Pin the path order remote sync reads files in
  galdre
  committed
  b442a03
- Drop Card, Dashboard and Document from the lint's schema check
  galdre
  committed
  d4a0c95
- Let a null default schema silence null-schema-reference
  galdre
  committed
  6405d92
Commits on Nov 2, 2026
- Quote names from the folder in setup error messages
  galdre
  committed
  fe64369
- Pin setup-bun and drop write credentials in Import
  galdre
  committed
  aa22915
- Name H2 as the Sample Database in the Import workflow
  galdre
  committed
  cbbfbbd
- Fail lint parity when it compares no transform or test
  galdre
  committed
  4b09079
- Share one --default-schema parser with the parity script
  galdre
  committed
  c77e48e
- Say no error rule predicts a refusal in parity
  galdre
  committed
  56e61c9
- Pin that DATABASE= gives a null default schema
  galdre
  committed
  c2dbdcb
- Find a YAML map's earlier keys without a linear scan
  galdre
  committed
  0f7ba14
- Refuse a YAML alias to a map or list that holds it
  galdre
  committed
  8a96973
- Read entity files through one reader for both commands
  galdre
  committed
  21ae98a
- Skip a file of an empty YAML list or string
  galdre
  committed
  49a9016
- Warn on a file that is not valid UTF-8 and lint its text
  galdre
  committed
  8f12704
- Report a symbolic link as an error without reading its target
  galdre
  committed
  32d7617
- Read no table or column from a field-filter tag
  galdre
  committed
  0ff119a
- Find a Python source table with a null table_id by name
  galdre
  committed
  8b5fef3
- Refuse a blank target schema on every engine
  galdre
  committed
  d7e9d2e
- Name a deleted source database as the transform's fault
  galdre
  committed
  1dd17d1
- Resolve entity ids through one lookup that names other models
  galdre
  committed
  60ba2f3
- List a file's databases with one function for both uses
  galdre
  committed
  d025072
- Pin that a Database file's null schema leaves references checked
  galdre
  committed
  64f4b46
- Pin that duplicate-target compares target databases
  galdre
  committed
  3d3c87e
- Check a transform's field-filter tables for null schemas
  galdre
  committed
  e81283e
- Compare ClickHouse column names with case
  galdre
  committed
  992b162
- Read a Postgres composite field as its column
  galdre
  committed
  f0ea420
- Explain a column read in another case and its fix
  galdre
  committed
  9cec378
- Make a dangling qualifier in an expectation an error
  galdre
  committed
  f91f068
- Report a quoted table name the test rewrite keeps
  galdre
  committed
  53dc2ef
- Pin stand-in keys for schema case and ten inputs
  galdre
  committed
  58697a2
- Pin that a field-filter field in another case needs no input
  galdre
  committed
  4d5d591
- Name SQLGlot only in reasons for SQL it cannot parse
  galdre
  committed
  3632ddc
- Name every case in five docstrings
  galdre
  committed
  8813c31
- Make three test names match their assertions
  galdre
  committed
  28e56b8
- Record a snapshot tree for each lint rule
  galdre
  committed
  691695e
- Give the fact each fix needs in four lint messages
  galdre
  committed
  5ce1898
- Correct the lint docs on remote sync, schemas, case and versions
  galdre
  committed
  854555a
- Describe everything the lint checks in one sentence
  galdre
  committed
  02f33d0
- Tighten the unknown-engine message, a docstring and comments
  galdre
  committed
  9ca1c6c
- Build table keys with one constructor per kind of name
  galdre
  committed
  7659322
- Walk each document's sources once per lint run
  galdre
  committed
  38a9805
- Keep the read table on each cycle edge, folding case on Redshift
  galdre
  committed
  a6b25b1
- Read a snippet tag's name with one parser
  galdre
  committed
  0ed8a0d
- Build lint test fixtures with the mutation library's helpers
  galdre
  committed
  4e38287
- Share small helpers and drop exports nothing imports
  galdre
  committed
  859565f
- Pin each mutation's findings by rule, file and related files
  galdre
  committed
  eea9425
- Run each mutation again as a remote-sync repository
  galdre
  committed
  e6ac718
- Drop column-check entries that change no output
  galdre
  committed
  46f6cee
- Pin column checks of aliased tables, mixed quoting and qualifiers
  galdre
  committed
  10ad71a
- Pin the duplicate-input list and the single-match suggestion
  galdre
  committed
  b7a4354
- Refer to the SQLite Sample Database with no schema
  galdre
  committed
  aa83af1
- State that the SQLite Sample Database runs no transform
  galdre
  committed
  f619cc2
- Say a field-filter tag names a table pull cannot find
  galdre
  committed
  6143191
- Merge main into the reference lint
  galdre
  committed
  fee070d
- Merge main into the reference lint
  galdre
  committed
  6a499ed

Commits

Commits on Oct 27, 2026

Commits on Oct 28, 2026

Commits on Oct 29, 2026

Commits on Oct 30, 2026