Antalya 26.3: DataLakeCatalog: avoid full catalog read for UNKNOWN_TABLE typo hints#1675
Conversation
…n_drop DataLakeCatalog: avoid full catalog read for `UNKNOWN_TABLE` typo hints
Verification report: Altinity/ClickHouse PR #1675ConclusionPR is merged. CI red on head, but every failure is either a pre-existing flake or a regression-suite scenario already broken at baseline on
CI on head
|
| Check | Test FAIL | Class |
|---|---|---|
Stateless tests (arm_asan, azure, parallel, 2/4) |
00084_external_aggregation |
Pre-existing flake — 28 / 12 PRs / 90d |
Stateless tests (arm_binary, sequential) |
00157_cache_dictionary |
Pre-existing flake — 106 / 25 PRs / 90d |
Regression workflow (10 failed checks)
| Check | Top failing tests on PR-1675 builds (30d) | Baseline (antalya-26.3, 30d) |
Class |
|---|---|---|---|
Swarms (Release + Aarch64) |
swarm sanity / cluster with only one observer node, swarm sanity / check scale up and down, swarm joins / join clause, swarm union (×4 each) |
30–49% on every PR | Pre-existing broken |
S3Export (partition) (Release + Aarch64) |
sanity / mismatched columns (×4) |
50% | Pre-existing broken |
Iceberg (1) (Release + Aarch64) |
rest catalog / sort key timezone / day transform positive utc offset, rest catalog / swarm / swarm examples (×4 each) |
41% / chronic | Missing-dep + pre-existing |
Iceberg (2) (Release + Aarch64) |
glue catalog / swarm / swarm examples (×4) |
chronic | Pre-existing |
Parquet (Release + Aarch64) |
postgresql/mysql round-trip compression-type variants (×4 each) | ~36% | Pre-existing flaky |
Regression DB on /PRs/1675/ builds (30d): 699 Fail / 44,007 OK ≈ 1.6%. Every top failure matches the all-PR baseline fail rate on antalya-26.3.
Related to PR diff?
PR is a 4-line guard in DataLakeCatalog — avoid scanning the full catalog when computing UNKNOWN_TABLE typo-suggestion hints (1 file changed). Touches only the error-path of getTable* for data-lake catalogs.
| Failing test | Diff overlap | Related? |
|---|---|---|
00084_external_aggregation, 00157_cache_dictionary |
none | No |
swarms / *, parquet / *, s3_export_partition / * |
none (data-lake-catalog typo hints ≠ any of these) | No |
iceberg / sort key timezone / * |
none — failure is UNRECOGNIZED_ARGUMENTS, missing-dep |
No |
iceberg / swarm / swarm examples (rest + glue) |
none — the change is on the typo-hint code path, not the swarm/iceberg integration | No |
No failing test intersects the typo-hint code path or fails above the all-PR baseline.
Local checkout
cd /Users/alsugilyazova/workspace/altinity-clickhouse/ClickHouse
gh pr checkout 1675 --repo Altinity/ClickHouse
# HEAD: 9d04f36ed7d7adbae5b146f8bee1389450392e1e
Audit Report — PR #1675Confirmed DefectsNo confirmed defects in reviewed scope. |
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Avoid scanning the whole remote data lake catalog for “Maybe you meant …” table hints when
show_data_lake_catalogs_in_system_tablesis disabled (ClickHouse#100452 by @alsugiliazova).Cherry-picked from ClickHouse#100452.
Documentation entry for user-facing changes
When
show_data_lake_catalogs_in_system_tables = 0, the server must not implicitly scan the whole remote data lake catalog.Previously, building the “Maybe you meant …” hint for a missing table in a
DataLakeCatalogdatabase still calledgetAllTableNames()→DatabaseDataLake::getTablesIterator(), which lists the entire catalog and loads per-table metadata—heavy work and can OOM on large catalogs, only to enrich an error message.This change makes
TableNameHints::getAllRegisteredNames()return an empty name list for data lake catalogs when that setting is off, so hint generation does not trigger a full catalog listing.Query examples
Drop non-existent table:
Previously (undesired): server could answer with
UNKNOWN_TABLEand a “Maybe you meant …” suggestion after scanning catalog names:After the fix:
UNKNOWN_TABLEwithout hint when the setting is0:Optional follow-up
If local hints are skipped for data lake + setting
0,getHintForTablecan still fall back togetExtendedHintForTable, which only scans non–data-lake databases. In edge cases a suggestion could point at another database’s table. If that is undesirable, a follow-up could skip extended hints under the same condition as local enumeration.CI/CD Options
Exclude tests:
Regression jobs to run: