fix: integration test fixes, bug fixes, and help text improvements#14
Open
yunqiqiliang wants to merge 6 commits into
Open
fix: integration test fixes, bug fixes, and help text improvements#14yunqiqiliang wants to merge 6 commits into
yunqiqiliang wants to merge 6 commits into
Conversation
… logic CDC prereq check: - Extract checkCdcPrereqs() as exported function in datasource.ts - Delete duplicate 95-line implementation from task.ts - check-cdc command now delegates to shared helper Duplicate task name check: - Extract checkDuplicateTaskName() helper in task.ts - Replace 6 identical 5-line inline blocks with single helper call Other cleanup: - Remove redundant targetDsType=1 reassignment in create/save-realtime-sync (was already initialized to 1) - Fix splitPkExclude: remove TINYINT/SMALLINT which were never in splitPkTypes - Replace 6 inline new Date().toISOString()... with formatIsoStartOfDay(undefined) Co-Authored-By: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
Bug fixes: - formatIsoStartOfDay(undefined) caused infinite recursion — now returns today's date - flow instances --instance conflicted with global --instance — renamed to --flow-instance - flow submit published:false — poll condition updated to accept deployStatus=3 (deployed with pending changes) - save-cron DDL guard: detect CREATE/DROP/ALTER TABLE in task content and block scheduling - cron-preview next_runs always empty (API returns []) — local fallback via cronNextRuns() - schedule-info on draft task gave cryptic error — now returns friendly TASK_NOT_DEPLOYED message - task search treated positional name as subcommand — changed to positional [name] - save-schedule hanging — caused by formatIsoStartOfDay infinite recursion (same root cause) - delete-folder non-empty error now lists contained tasks by name - task create validates name characters before calling API Help text improvements: - task flow: describe composite task concept (DAG of nodes) and typical workflow - All flow subcommands: add context, usage order, and cross-references - node-save --content/--file: add missing describe fields - task create --type: fix trailing quote, add FLOW explanation - task create --folder: mark as demandOption, improve description - All <task> positionals (24 instances): add "Task name or ID" describe - create-offline-sync/save-offline-sync: remove agent/MCP jargon - offline-sync-schema: remove "Agent uses output" phrasing - create-realtime-sync/create-batch-sync: remove internal type names - task start --startup-mode: translate Chinese to English - task start --config: translate Chinese values to English - task start --engine-type/--blacklist-strategy: replace circular descriptions - save-schedule --retry-unit/--timeout-unit: clarify m=minutes s=seconds - task list/list-folders --page/--page-size: add describe fields - flow node-save-config --cron/--vc/--schema: add describe fields - Confirmations unified to "Requires -y to proceed." Chinese aiMessage → English: - task execute: translate completion message - task list: translate pagination hint - task list-folders: translate pagination hint - task search: remove Chinese truncation suffix New feature (cron-adapter.ts): - cronNextRuns(): local Quartz cron calculator supporting named days/months, ranges, steps — used as fallback when API returns empty next_runs Co-Authored-By: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
The bind API returns success before the edge is durably written to DB. Add post-bind verification: re-query DAG up to 3 times (800ms apart). If edge is missing, retry the bind call automatically. If still not confirmed after 3 attempts, return a warning instead of false success. Co-Authored-By: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
…ist; note flow submit known issue Target table guard: - save-offline-sync: check if sink table exists in Lakehouse before saving field mapping config. If missing, abort with TARGET_TABLE_NOT_EXISTS (exit code 2) and output CREATE TABLE IF NOT EXISTS DDL. If table exists, suppress DDL field and warning from response. - task deploy: same check as second guard — parses saved fileContent to verify sink table existence before publishing. - listDataObjects returns string[] (table names), fixed parsing accordingly. Tested: - save-offline-sync with existing table: passes, no DDL in response - save-offline-sync with missing table: blocked with DDL output (exit 2) - deploy with existing table: deploys successfully (status: online) Flow submit known issue: - submit command description and all related aiMessages now note that CLI submit may not fully publish — user should verify in Studio UI and click Submit manually if flow status does not change to Published. Co-Authored-By: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
…e/save-realtime-sync checkCdcPrereqs: Oracle (dsType=25) now returns ok:false with a clear "not supported" message instead of silently passing through. create-realtime-sync / save-realtime-sync: validate source dsType against the supported CDC set before proceeding. Supported sources: MySQL/TiDB/MariaDB (5,17,18,19,39), PostgreSQL/Greenplum (7,22,40,46,48), SQL Server (8), DM (26). Unsupported types (including Oracle) are rejected with UNSUPPORTED_DATASOURCE (exit code 2). Co-Authored-By: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
…n, collectType datasource check-cdc: - SQL Server: replace sys.databases/sys.dm_server_services queries (inaccessible via getSampleData API) with INFORMATION_SCHEMA.TABLES filtered client-side for TABLE_SCHEMA='cdc'. If cdc schema tables exist, CDC is enabled and Agent is inferred running. - Oracle (dsType=25): now returns ok:false with explicit "not supported" message instead of silently passing through. create-realtime-sync / save-realtime-sync: - Added CDC_SUPPORTED_TYPES whitelist. Supported: MySQL/TiDB/MariaDB, PostgreSQL/Greenplum, SQL Server, DM. Oracle and others rejected with UNSUPPORTED_DATASOURCE (exit code 2). save-offline-sync: - INTEGRATION tasks now use collectType:1 (was 0 from saveTaskContent). collectType:0 caused Studio UI to treat the task as a script task, leaving field mapping switches in wrong initial state. - SQL Server: added FlinkX unsupported type check before saving. Blocked types: sql_variant, image, binary, varbinary, timestamp. text/ntext are supported (map to STRING). Error includes fix hint. offline-sync-schema: - SQL Server note updated: lists unsupported types and clarifies text/ntext map to STRING. Co-Authored-By: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Full integration test pass across all 57 commands (datasource, task, task flow). Fixed bugs found during testing and improved help text quality throughout.
Bug Fixes
save-schedulehanging indefinitelyformatIsoStartOfDay(undefined)caused infinite recursion — returns today's date nowflow instancesparameter conflict--instanceconflicted with global--instance— renamed to--flow-instanceflow submitalways returnspublished: falsedeployStatus=3(deployed with pending changes)save-cronallows DDL tasks to be scheduledCREATE/DROP/ALTER TABLEwithDDL_TASK_SCHEDULE_FORBIDDENcron-previewalways returns emptynext_runs[]on server — added localcronNextRuns()fallback incron-adapter.tsschedule-infoon draft task gives cryptic errorTASK_NOT_DEPLOYEDmessagetask search demotreated as unknown subcommand--nameoption to positional[name]delete-foldererror doesn't say what's insidetask createwith/in name gives cryptic API error"Requires -y to proceed. No action was executed."Help Text Improvements
task flow: Explains composite task concept (DAG of SQL/Python/Shell nodes) and full workflowtask create --type: Fixed trailing quote; FLOW now described as "composite task"task create --folder: Marked asdemandOption, improved description<task>positionals: Added"Task name or ID"describe fieldcreate-offline-sync/save-offline-sync: Removed agent/MCP jargoncreate-realtime-sync/create-batch-sync: Removed internal type names (MULTI_REALTIME, MULTI_DI)task start: Translated Chinese option descriptions to English--retry-unit/--timeout-unit: Clarifiedm=minutes, s=secondsflow node-save-config --cron/--vc/--schema: Added missing describe fieldsChinese → English
task executecompletion aiMessagetask list/task list-folderspagination hinttask searchtruncation suffixNew:
cronNextRuns()(cron-adapter.ts)Local Quartz cron calculator supporting named days/months (MON-FRI, JAN-DEC), ranges, and steps. Used as fallback when the server API returns empty
next_runs.Testing
Full regression test: 57/57 commands pass (datasource, task CRUD, sync, flow subcommands).