Support managed per-job Borg exclusion pattern files with --exclude-from

[borg-codex-bot]

Summary

Add optional per-job support for Borg exclusion pattern files through --exclude-from PATH.

A central exclusion file lets experienced Borg users maintain many file and directory rules in one place. It complements the current explicit path exclusions and the proposed marker-based --exclude-if-present option without requiring arbitrary additional Borg arguments.

Motivation

Users migrating existing Borg scripts often already maintain an exclusion file. Without --exclude-from, every rule must be recreated as an individual path in the wizard. This is inefficient for large or dynamic configurations and prevents reuse of Borg-native pattern styles.

Current behavior

Borg Backup UI currently stores concrete exclusion paths in each job and generates a separate Borg --exclude argument for each path. It does not accept a Borg pattern file and intentionally does not expose a free-form command-line field.

Borg semantics

--exclude-from PATH reads exclusion patterns from a file. The file can contain Borg pattern styles and therefore has more expressive behavior than the current absolute path picker.

The implementation should preserve Borg's native file semantics instead of translating the file into UI paths. Explicit path exclusions, marker-based exclusions and exclusion files should be additive.

Reference: https://borgbackup.readthedocs.io/en/1.4-maint/usage/create.html

Implementation alternatives

Option A: Reference an existing server-side file

Store a path to an existing exclusion file and pass it to Borg at runtime.

Advantages:

• smallest implementation;
• users can continue editing their existing file;
• no duplicate copy is created.

Disadvantages and risks:

• the file can be moved, deleted or become unreadable;
• its contents can change outside the job wizard without an auditable job edit;
• it may not be included in plugin settings backup or job export;
• a path on temporary storage may disappear after reboot;
• importing the job on another system can leave a broken reference;
• unrestricted paths create avoidable validation and support complexity.

If this option is selected, the implementation must validate the path as a readable regular file at save time and again immediately before borg create. A missing or unreadable file must fail the job before archive creation. The allowed persistent path roots must be defined explicitly; arbitrary raw paths must not be forwarded unchecked.

Option B: Import a managed copy into the plugin configuration

Let the user select or upload an exclusion file and copy its exact contents into the existing persistent plugin data layout. The job references the managed copy rather than the original file.

Advantages:

• stable behavior across reboots;
• the exact rules used by the job are owned by the job configuration;
• settings backup/restore and job transfer can include the file deliberately;
• external edits cannot silently change a backup job;
• missing-file diagnostics and lifecycle handling are easier to define.

Disadvantages and risks:

• replacing or removing the file needs an explicit lifecycle;
• job duplication, deletion and immutable job IDs must be considered;
• the UI must show which copy is active and when it was imported;
• imported content can disclose filesystem names in exports or support packages and must be handled consistently with existing job paths;
• size and encoding limits are required.

This is the recommended first implementation because it makes backup behavior reproducible and keeps the job self-contained.

Option C: Add an integrated Borg pattern editor

Store the pattern text with the job and provide an editor with validation and documentation.

Advantages:

• best discoverability;
• no external files;
• easy review while editing the job.

Disadvantages and risks:

• largest UI and validation scope;
• Borg pattern syntax is expert functionality and easy to misunderstand;
• syntax highlighting, line-level errors and safe editing increase complexity;
• changing Borg syntax across supported versions would need consideration.

This can be a later enhancement. It should not block a managed file import.

Recommended phased design

Phase 1: Managed import

Add an advanced subsection under Job Wizard -> Sources and target -> Exclusions:

• Optional exclusion pattern file.
• Import or replace a file explicitly.
• Show the original filename, import timestamp, size and a content hash.
• Allow the managed file to be downloaded for inspection and removed with confirmation.
• Explain that patterns use Borg syntax and can exclude more data than intended.
• Do not provide or accept arbitrary additional Borg flags.

Store the exact imported bytes in the established persistent plugin data layout. The precise path must follow current job storage conventions and must not introduce a new top-level data architecture.

Suggested metadata shape:

{
  "exclude_from": {
    "managed_file": "relative-managed-reference",
    "original_name": "exclude.txt",
    "sha256": "...",
    "imported_at": "..."
  }
}

The final schema should align with the immutable job ID work in #447. If this feature is implemented before #447, its storage design must avoid adding another filename keyed only by mutable job names or archive prefixes.

Phase 2: Optional editing

After the managed import is stable, consider a dedicated editor for the managed content. Editing must be explicit, validated and visible as a job configuration change.

Runtime behavior

1. Resolve only the managed reference belonging to the current job.
2. Verify that it is a readable regular file and still matches the stored metadata or update policy.
3. Fail before borg create if the file is missing, unreadable, oversized or invalid according to the selected validation level.
4. Add arguments without shell interpolation:

--exclude-from /resolved/managed/path

5. Combine the file with existing explicit --exclude arguments and optional --exclude-if-present markers.
6. Log the managed filename and hash, but do not dump the full rule contents into normal logs.
7. Show the active file in the flow preview and final job summary.

Validation and safety requirements

• Define a conservative maximum file size and maximum line length.
• Reject NUL bytes and unsupported file types.
• Decide whether UTF-8 is required or whether the exact byte content is retained.
• Never evaluate the file through a shell.
• Do not silently ignore missing or unreadable files.
• Do not silently rewrite Borg patterns.
• Provide a prominent warning that an incorrect rule can exclude required backup data.
• Encourage a manual backup run and archive inspection after adding or replacing the file.
• Preserve file permissions appropriate for plugin-owned configuration.
• Ensure path traversal and symlink attacks cannot escape the managed storage location.
• Define cleanup behavior when a job is deleted, while respecting repository and migration safety rules.

Import, export and diagnostics

The implementation must define:

• whether the managed file is included in job export and full settings backup;
• how filename collisions are avoided;
• how job duplication creates an independent or shared copy;
• how restore handles a missing, corrupt or conflicting managed file;
• how support packages report metadata without unnecessarily exposing the full pattern list;
• how system health reports an orphaned or missing managed file;
• how a future immutable job ID migration maps the file safely.

Open decisions

Before implementation, decide:

1. Managed import only, external reference only, or both?
2. One exclusion file per job initially, or a repeatable list?
3. Should the browser select a file already on the Unraid server, upload one from the administrator's computer, or support both?
4. Should Borg be the only syntax validator, or should the UI perform limited structural validation first?
5. Should the managed file be included in job transfer by default?
6. How should existing managed files be associated with jobs before and after Introduce immutable job IDs across all job-dependent data and workflows #447?

Tests

Add coverage for:

• import, replacement, removal and persistence;
• exact byte/hash handling;
• missing, unreadable, oversized and malformed files;
• path traversal and symlink rejection;
• exact Borg argument construction without shell interpolation;
• coexistence with explicit paths and marker exclusions;
• job duplication, deletion and transfer behavior;
• backup/settings restore behavior;
• German and English wizard text and documentation;
• a focused runtime test proving that patterns from the managed file affect archive contents as expected;
• unchanged behavior for every job without an exclusion file.

Acceptance criteria

• A user can attach an optional Borg exclusion pattern file to a job through a defined, safe lifecycle.
• The file is passed to Borg with --exclude-from without shell evaluation or silent rewriting.
• Backup creation fails clearly before archive creation when the configured file cannot be used.
• Existing explicit exclusions remain functional and additive.
• Existing jobs without this option remain unchanged.
• Import/export, backup/restore, diagnostics and future job ID handling are explicitly covered.
• German and English UI text, documentation, tests and a release note are included.

Dependency on #447

This issue is intentionally blocked by #447 and should be implemented only after immutable job IDs are available.

The recommended managed-import design creates a persistent, job-owned exclusion file. Its storage path, lifecycle, duplication, cleanup, transfer and recovery behavior must be keyed by the immutable job ID introduced in #447. Implementing the managed file before #447 would require a temporary association based on a mutable job name, job key or archive prefix and would then require another migration. It would also increase the risk of orphaned files or incorrect ownership after a job is renamed or otherwise changed.

Required implementation order:

1. Complete and merge Introduce immutable job IDs across all job-dependent data and workflows #447, including the immutable job ID migration and job-dependent storage contract.
2. Define the managed exclusion file location and lifecycle using that immutable job ID.
3. Implement the managed --exclude-from workflow described in this issue.

The external-path-only alternative described under Option A could technically be implemented without #447 because it stores only a path in the job metadata. However, that is not the recommended target design and must not be used merely to bypass this dependency unless the issue scope is explicitly changed.

Until #447 is complete, this issue remains blocked and implementation should not begin.