feat(import): import several files or a folder in one run by datlechin · Pull Request #3745 · TableProApp/TablePro

datlechin · 2026-10-21T06:14:39Z

Summary

File > Import > Import Data… takes several files or a folder. One file still opens its usual sheet. A folder brings in the visible files directly inside it that load into a table; anything else, such as a .sql dump, is named in the sheet and left out.
New sheet, RowImportBatchSheet with its model RowImportBatch. Each file starts on the existing table its name matches, else a new table named as the one-file sheet names it, with no two files on the same new name.
Each row has an include checkbox, a destination menu, a mapping summary and Edit…, which opens the one-file sheet in an edit mode from the table's saved mapping. Files or folders dropped on the list are added.
Options are set once per format. Mixed formats must agree on On error and Wrap in transaction, read through a new PluginKit property, ImportFormatPlugin.rowImportConfiguration. Additive (defaults to nil); the ABI check passed and the kit version stays 40.
Delete existing rows is refused for a table that two files of the run fill, new or existing.
ImportService.importFiles runs every file in one lease, one session scope and one transaction. Each file's writer joins it and sends no BEGIN, COMMIT or ROLLBACK of its own (ImportDataSinkAdapter(joinsTransaction:)).
Stop and Rollback rolls back every file. Stop and Commit keeps what already ran. Skip and Continue moves past a failed file. Pressing Stop rolls back. History records only rows that stayed.
Table creation and authorization move out of RowImportSheet into RowImportTableMaker, which asks Safe Mode once for every CREATE TABLE and retry DELETE in the run. The row inserts reuse that grant.
Summary alert: one line per file with its table and row count, including rows a failed or stopped file left behind. Copy Details carries every skipped row's error and the full text of a failed file's error.
Fix: Stop pressed while a new table was being created was ignored, in the one-file sheet too.

Not in this PR

Per-file parsing options. Each format plugin reads one settings object for the whole run, so a per-file delimiter or encoding needs a PluginKit change. Auto-detect already reads delimiter and encoding per file.
New tables inside the transaction. They are created before the run's transaction starts, so a rolled-back run leaves them empty, as the one-file sheet does; a retry reuses them.
Older bug: after a failed statement on PostgreSQL the app still sends COMMIT, which PostgreSQL treats as ROLLBACK, so Stop and Commit's "kept" is wrong there, for one file and for several.
Older bug: the Stop confirmation says statements "stay committed" even when the import's transaction rolls them back.
Dropping a folder on the connection window still opens files as data files.
No screenshots and no UI test: the only way into the sheet is the open panel.

Test plan

New ImportServiceBatchTests: one transaction for every file, a later failure rolls back earlier files under Stop and Rollback, Stop and Commit, Skip and Continue, a run without a transaction, joining the user's transaction, Stop before the first file, history after a rollback.
New RowImportBatchTests: table matching and new-table names, twenty files into twenty new tables, duplicate names, mixed format options, Delete existing rows refusals, unreadable files, mapping by name, retries.
New ImportBatchFilesTests (folder contents and order) and RowImportConfigurationTests (CSV, JSON, XLSX report their options). ImportDataSinkAdapterRollbackTests: a joining sink sends no transaction statements.
87 tests, build, lint, l10n and ABI checks passed locally, including a 20-file run where file 12 fails and Stop and Rollback rolls back all of them.

Fixes #2499

…existing

…-files

J2TeamNNL

Automated code review: 10 findings.

J2TeamNNL · 2026-10-21T09:28:10Z

        return nil
    }

+    /// A folder holding a single file a row format reads opens the sheet for one file. Files no row format reads


presentRowImportBatch opens the batch sheet (or the one-file sheet for a single-file folder) without the eligibility check presentRowImport runs via rowImportRefusal. On a read-only connection, an unsupported connection type, or with a sheet already presented, the user configures the whole batch and is only refused at Import. Call rowImportRefusal for the resolved files' formats first.

J2TeamNNL · 2026-10-21T09:28:10Z

+    }
+
+    @MainActor
+    private func runFiles(


A failure from beginTransaction in runFiles, or from the final commit in endTransaction, is thrown without the failure history row or completion report that single-file importFile makes. If the last COMMIT fails after 20 files, the user gets a generic alert with no history entry or notification.

J2TeamNNL · 2026-10-21T09:28:10Z

+    }
+
+    /// A file's row is recorded only while its rows are still there, so a rolled-back run records the failure alone.
+    private func recordHistory(of report: RowImportBatchReport, in scope: DatabaseScope) async {


recordHistory skips .stopped outcomes. With wrapInTransaction off or Skip and Continue, pressing Stop in file 3 after 5,000 rows leaves those rows in the database (rowsKept is true) but case .imported, .stopped, .notRun: break records no history row for them.

J2TeamNNL · 2026-10-21T09:28:10Z

+            state.isImporting = false
+            currentProgress = nil
+        }
+        guard !isStopRequested else {


The early-return stopped report uses transaction .untracked, so the alert says earlier files were kept. RowImportTableMaker.prepare never checks the stop flag, so Stop during table creation creates every table, then reports 'Stopped at X. The files imported before it were kept.' although nothing was imported and empty tables remain.

J2TeamNNL · 2026-10-21T09:28:10Z

+    }
+
+    @MainActor
+    private func runFile(


Progress updates for the previous file can overwrite the next file's reset state. track() queues Task { @MainActor } from KVO callbacks; after file N finishes and observers are invalidated, queued Tasks can run after runFile(N+1) resets progress, writing file N's final counts over the reset.

J2TeamNNL · 2026-10-21T09:28:10Z

+        return nil
+    }
+
+    private func policyProblem(in included: [File]) -> String? {


A plugin whose rowImportConfiguration is nil (default for third-party plugins built before this requirement) makes policyProblem return '%@ files cannot be imported together with other files.' for any multi-file run, with no way to fix it in the sheet. A folder of those files cannot be imported at all although the panel's readsRows accepts them.

J2TeamNNL · 2026-10-21T09:28:10Z

+            Button("Cancel") { isPresented = false }
+                .keyboardShortcut(.cancelAction)
+            Button("Import") { performImport() }
+                .buttonStyle(.borderedProminent)


The Import button's disabled state reads importService?.state.isImporting, but importService is plain @State, so the view does not re-render when the service changes. While prepare creates tables isImporting is false, so a second click or Return starts a second performImport that replaces the service and races on the same tables. Add a re-entry guard.

J2TeamNNL · 2026-10-21T09:28:10Z

+        .frame(maxWidth: .infinity, alignment: .leading)
+    }
+
+    /// A dropped file or folder joins the run, read the same way as a folder chosen in the panel.


onDrop is not disabled during a run. A file dropped mid-import mutates batch.files and triggers reads and detectFields while the captured plan executes, and after a stopped or failed run the dropped file looks part of the run but is not in the report.

J2TeamNNL · 2026-10-21T09:28:10Z


    var fieldDetectionSignature: String { settings.detectionSignature }

+    var rowImportConfiguration: RowImportRunner.Configuration? { runConfiguration }


rowImportConfiguration and runConfiguration are pasted identically into the CSV, JSON and XLSX plugins, where rowImportConfiguration just returns runConfiguration. A new setting forgotten in one copy silently desynchronises batch policy from a file's own import. Share one definition.

J2TeamNNL · 2026-10-21T09:28:11Z

@@ -0,0 +1,653 @@
+//


Added docblocks break the repo rule 'Code comments say why, never what' (AGENTS.md). Examples: /// What the sheet that edits one file handed back..., /// Files and folders dropped on the list join the run..., /// Tables alone, because they are the only objects.... Several narrate design history. Trim to why-only comments.

J2TeamNNL · 2026-10-21T09:31:26Z

Fixes for the review findings are in J2TeamNNL@ed43d48.

Fixed (10 of 10):

presentRowImportBatch now runs rowImportRefusal for each format before opening the sheet.
A failure from begin or the final commit in importFiles now writes a failed history row and the completion report.
A Stop that leaves rows behind now records a history row for the stopped file.
A Stop before any row is written now says "No rows were imported" instead of claiming earlier files were kept.
Progress updates are ignored unless they come from the file now running, so a late update cannot overwrite the next file's reset.
A plugin with no rowImportConfiguration now batches with the default policy (stop and roll back, in a transaction) instead of being refused.
The Import button and Return are guarded by a run flag, so a second run cannot start while tables are being created.
Drops are ignored while a run is going.
CSV, JSON and XLSX share one Configuration init through a new RowImportOptions protocol.
Docblocks that restated the code are removed from RowImportBatch and RowImportBatchSheet.

Skipped: none.

Not verified: no local build, and swiftlint cannot load sourcekit here. The files parse with swiftc. The new alert string is not yet in the String Catalog.

Cherry-pick:
git fetch https://github.com/J2TeamNNL/TablePro review-fix/3745 && git cherry-pick ed43d487084802c38066086e30196be54919ae60

…-files # Conflicts: # apple/TablePro/Views/Main/MainContentView.swift

datlechin added 3 commits October 21, 2026 12:07

feat(import): import several files or a folder in one run

c10f5de

chore(l10n): add the strings for importing several files

a119a78

docs(changelog): note importing several files

4d32605

datlechin mentioned this pull request Oct 21, 2026

docs(features): import several files or a folder in one run TableProApp/docs#97

Merged

datlechin added 2 commits October 21, 2026 13:29

fix(import): refuse clearing a table two files of a run fill, new or …

e8e6537

…existing

Merge remote-tracking branch 'origin/main' into feat/2499-import-many…

09248f4

…-files

datlechin enabled auto-merge (squash) October 21, 2026 07:49

J2TeamNNL reviewed Oct 21, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into feat/2499-import-many…

b67414c

…-files # Conflicts: # apple/TablePro/Views/Main/MainContentView.swift

datlechin merged commit fbf13dd into main Oct 21, 2026
8 checks passed

This was referenced Oct 21, 2026

feat(import): import rows as Update, Insert or Update, or Insert, Skip Existing #3743

Merged

Import several CSV files in one pass #2499

Closed

datlechin deleted the feat/2499-import-many-files branch October 22, 2026 07:05

Navigation Menu

Repository navigation

feat(import): import several files or a folder in one run - #3745

feat(import): import several files or a folder in one run#3745
datlechin merged 6 commits into
mainfrom
feat/2499-import-many-files

datlechin commented Oct 21, 2026 •

edited

Loading

Uh oh!

J2TeamNNL left a comment

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		var fieldDetectionSignature: String { settings.detectionSignature }

		var rowImportConfiguration: RowImportRunner.Configuration? { runConfiguration }

Conversation

datlechin commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Not in this PR

Test plan

Uh oh!

J2TeamNNL left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

J2TeamNNL commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

datlechin commented Oct 21, 2026 •

edited

Loading