fix: protect library files and release 0.2.25

.github/workflows/build-macos.yml

@@ -14,7 +14,10 @@ permissions:
   contents: write   # upload de l'asset sur la release
 
 jobs:
+  tests:
+    uses: ./.github/workflows/tests.yml
   build:
+    needs: tests
     name: DDD.app (Apple Silicon)
     runs-on: macos-14
     steps:

.github/workflows/tests.yml

@@ -0,0 +1,31 @@
+name: tests
+
+on:
+  pull_request:
+  push:
+    branches: [master]
+  workflow_dispatch:
+  workflow_call:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [windows-latest, ubuntu-latest, macos-14]
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - name: Install test dependencies
+        run: python -m pip install -e ".[gui,test]"
+      - name: Run all tests (core, CLI, GUI and legacy scenarios)
+        run: python -m pytest -q
+      - name: CLI smoke test
+        run: python -m ddd --help

README.md

@@ -29,10 +29,10 @@
 **DDD cleans up your music library and bumps it to club-playable quality, on its own.**
 
 You point it at a folder (or your Discogs / Bandcamp favorites), and DDD:
-- spots the **fake lossless**: MP3s re-encoded as .flac/.wav/.aiff that *look* lossless but aren't,
-- goes and finds a **real version** on Soulseek (FLAC, WAV or AIFF, with an automatic MP3 320 fallback if nothing lossless turns up),
-- checks the **spectrum** of the downloaded file to make sure it holds up (not an upscale), that it's the **right track** and not a snippet,
-- files it into **a single** clean library, and sends the fakes to the trash.
+- flags **limited bandwidth and suspicious encodings** for review,
+- searches for a **candidate version** on Soulseek (FLAC, WAV or AIFF, with an automatic MP3 320 fallback if nothing lossless turns up),
+- checks the downloaded file against your **bandwidth threshold**, requested identity and duration,
+- adds accepted candidates to your library and **retains originals by default**.
 
 No need to be a developer: download the `.exe`, double-click, it's a window.
 
@@ -57,26 +57,28 @@ No need to be a developer: download the `.exe`, double-click, it's a window.
 ## What it does
 
 - **Quality scan**: every file is ranked by its **spectral cutoff** (the frequency where the sound stops) into one of four bands, plus duplicates:
-  - **Lossless** (green): full spectrum, real lossless.
+  - **Wide** (green): wide spectrum observed; compression history remains unknown. The CSV identifier `LOSSLESS` is retained for compatibility.
   - **HQ** (blue): >= 18 kHz, playable on a big system (includes MP3 320).
   - **Iffy** (yellow): 16-18 kHz, borderline.
-  - **Bad** (red): < 16 kHz, mush.
+  - **Review** (red): reduced bandwidth or bitrate below the configured floor.
 - **Reads pretty much anything**: FLAC, WAV, AIFF, MP3, OGG/Opus, **MP4 / M4A** (AAC *and* ALAC), **WMA** (including WMA Lossless), APE, TTA, WavPack. That old folder of iTunes-era `.mp4` rips or Windows Media `.wma` gets scanned like the rest, instead of showing up empty. What's *inside* the container decides: an ALAC `.m4a` is treated as lossless and put through the same anti-upscale spectral check as a FLAC, not written off as lossy because of its extension.
 - **Quality / target modes** (in Settings) - the bar DDD keeps to, and what it goes hunting for:
   - **DJ Club** (>= 18 kHz) - *default*: keeps anything club-playable, MP3 320 included.
   - **Audiophile** (>= 20 kHz): rejects MP3s below 320.
-  - **Purist** (pure lossless): real full-spectrum lossless only; if it's not on Soulseek -> buy links, no MP3 fallback.
+  - **Purist** (wide spectrum): only candidates classified as wide spectrum; if it's not on Soulseek -> buy links, no MP3 fallback.
   - **MP3 320** (vintage / mobile): hunts MP3 320 straight, skips FLAC - for old gear that won't read FLAC, or syncing over mobile data. Bumps your sub-320s up to 320; leaves the lossless you already have untouched.
   - **WAV/AIFF only** / **FLAC only**: target a single lossless container (old CDJs/samplers that won't read FLAC, or a FLAC-homogeneous library). DDD never transcodes what you already own - the mode just picks the format of what it fetches.
-- **Upgrade**: replaces your below-the-bar files with something better, found on Soulseek. In **DJ Club / Audiophile** it looks for FLAC, WAV and AIFF (lots of DJs share in WAV/AIFF), with an **automatic MP3 320 fallback** for tracks that can't be found in lossless. MP3s below 320 kbps are **banned across the board**, whatever the mode.
+- **Upgrade**: downloads candidates from Soulseek and retains the originals by default. The CLI `--trash-original` option or GUI checkbox explicitly enables removal, only after a verified copy has been installed. Name collisions retain both names using a numbered suffix. In **DJ Club / Audiophile** it looks for FLAC, WAV and AIFF (lots of DJs share in WAV/AIFF), with an **automatic MP3 320 fallback** for tracks that can't be found in lossless. MP3s below 320 kbps are **banned across the board**, whatever the mode.
 - **Get favorites**: scrapes your Discogs wantlist / Bandcamp wishlist and downloads it.
 - **YouTube set / playlist**: paste a set URL (YouTube / 1001Tracklists) or a **YouTube playlist** (each video = a track) -> DDD extracts the tracklist into a want-list (CSV).
-- **Single library**: everything that passes lands in `~/Music/DDD` (changeable in Settings), de-duplicated. Rejects go to the **trash** (recoverable), never hard-deleted.
+- **Single library**: everything that passes lands in `~/Music/DDD` (changeable in Settings), de-duplicated. Rejected download candidates go to the **trash**. Import leaves rejected, unreadable and duplicate source files in place.
 - **Sort by genre** (*Sort by genre* button / `ddd sort`): files your loose tracks into your own **vibe folders** through a cascade - the file's **ID3 genre tag**, then **Discogs** (+ MusicBrainz), and when both come up empty a **local audio model** (Discogs-EffNet, 400 Discogs styles) that reads the genre **from the spectrum** - so even an untagged, badly-named edit (`Track_01.flac`) lands in the right folder instead of `_INBOX`. The default set is house/techno-oriented (ACID, DEEPWATER, HOUSERZ, PROG, TECHNO, TRANCE, GARAGE, DISCO-FUNK, BREAKS-ELECTRO) and is **fully editable** in Settings. **Dry-run by default** - you preview, then Apply. Only loose files are touched, never your curated subfolders. (The audio model runs **on-device, no cloud**; by the MTG/UPF, CC BY-NC.)
 - **Identify - recover lost names** (*Identify* tab / `ddd identify`): files called `YH1`, `track01`, `unknown 04`? DDD reads each file's **acoustic fingerprint** (Chromaprint) and matches it against **AcoustID** (the open, MusicBrainz-backed database, same idea as Shazam) to recover *Artist - Title*. It **proposes**, you **confirm track by track** (confident matches pre-ticked, uncertain ones flagged), then it renames + tags only what you kept - **never a blind rename**. Underground / unreleased tracks that aren't in the database just stay untouched.
 - **Not found -> buy links**: whatever Soulseek can't find comes out as a clickable page (DDD logo + theme) with **Discogs** (vinyl marketplace, perfect for old pressings) and **Bandcamp** links to buy it.
 
-**The safety net: the spectrum is law.** Every download is re-audited by spectrum (FFT); **the declared format and bitrate are only used for the Soulseek search, never for the keep-or-reject decision.** The spectrum doesn't lie, tags do - that's what tells a real 320 / lossless apart from an upscale (an MP3 128 re-encoded as .flac or .wav, which Soulseek's filters can't see). A file is only kept if it passes three checks: **spectral** (above the preset's bar, not an upscale), **duration** (not a snippet / preview) and **title + artist identity** (the right track, not a wrong match). Otherwise -> trash.
+**Safety and limits.** Spectral analysis measures bandwidth; it cannot prove the history of a recording. A genuine filtered recording can fall below your chosen threshold. Review by listening before removing originals. Downloads must match the complete normalized title, a complete named collaborator, the requested version and measured duration (10% tolerance, minimum 2 seconds). Without a reference duration, candidates under 90 seconds are rejected; unknown duration or artist is not accepted. Matching is conservative and can reject valid naming variations.
+
+Accepted downloads are copied without overwriting existing files, flushed and checked by SHA-256 before staging is removed. Original removal is opt-in and happens afterwards. Import never trashes source files; importing a library into itself is a no-op. Scan and rename only label content-identical files as duplicates; import retains such copies for review.
 
 ## Getting started (user)
 
@@ -91,7 +93,7 @@ No need to be a developer: download the `.exe`, double-click, it's a window.
    **Get favorites** tab: pick Discogs/Bandcamp -> *Fetch & download*.
 
 > The 3 D's: **DIG** your sources -> **DOWNLOAD** from Soulseek -> **DETECT** by spectrum. The output
-> is your verified lossless library, which you can then share / point anywhere you want.
+> is a library of candidates checked against your selected thresholds, which you can then share / point anywhere you want.
 
 ## Usage & responsibility
 
@@ -127,10 +129,10 @@ playlists (yt-dlp). Everything is bundled into the `.exe` (no Python, no ffmpeg
 # Scan a folder: Lossless / HQ / Iffy / Bad? well named? duplicates?
 .\.venv\Scripts\python.exe -m ddd scan "C:\path\to\Music"
 
-# Upgrade: drops real lossless into the library, fake source -> trash
+# Upgrade: add accepted copies; retain originals (use --trash-original to opt into removal)
 .\.venv\Scripts\python.exe -m ddd upgrade "C:\path\to\Music"
 
-# Import an existing folder into the library (lossless kept, the rest trashed)
+# Import accepted files; leave rejects, errors and confirmed duplicates at source
 .\.venv\Scripts\python.exe -m ddd import "C:\path\to\Music"
 
 # Rename a folder back to "Artist - Title" (from name + tags; dry-run, --apply to write)
@@ -164,10 +166,21 @@ Query resolution: name `Artist - Title` -> else ID3/Vorbis tags -> else title-on
 Compilations (`Various Artists`), vinyl side prefixes (`A1`, `B2`...) and artists duplicated
 in the title are normalized before the search.
 
+### Verification
+
+```sh
+python -m pip install -e ".[gui,test]"
+python -m pytest -q
+```
+
+The suite includes the standalone legacy scenarios in isolated subprocesses. CI runs core,
+CLI and GUI construction tests on Windows, Linux and macOS for pull requests. The macOS
+release build depends on this suite; the Windows build script also runs it before packaging.
+
 ### Building the `.exe`
 
 ```powershell
-.\.venv\Scripts\python.exe -m pip install -e ".[gui,build]"
+.\.venv\Scripts\python.exe -m pip install -e ".[gui,build,test]"
 .\packaging\build.ps1
 ```
 
@@ -189,7 +202,7 @@ docker run --rm -v /mnt/music:/music ddd scan /music -o /music/ddd-scan.csv
 docker run --rm \
   -e DDD_SOULSEEK_USER=you -e DDD_SOULSEEK_PASS=secret \
   -v /mnt/music:/music \
-  ddd upgrade /music --download-dir /music --apply
+  ddd upgrade /music --download-dir /music
 ```
 
 x86_64 only. Details: `docker/README.md`.

ddd/__init__.py

@@ -4,4 +4,4 @@
 dig le spectre du fichier. Coeur Python portable (CLI + futur GUI).
 """
 
-__version__ = "0.2.24"
+__version__ = "0.2.25"

ddd/cli.py

@@ -27,7 +27,7 @@
 # Ordre d'affichage du resume (du plus actionnable au moins)
 VERDICT_ORDER = [LOSSLESS, HQ, DOUTEUX, MAUVAIS, "ERROR", "SKIPPED"]
 VERDICT_LABEL = {
-    LOSSLESS: "lossless (full spectrum)",
+    LOSSLESS: "wide spectrum (compression history unknown)",
     HQ: "HQ (club-playable, >=18 kHz)",
     DOUTEUX: "iffy (16-18 kHz)",
     MAUVAIS: "bad (<16 kHz)",
@@ -144,12 +144,14 @@ def progress(*a) -> None:
             print(f"  scan [{a[0]}/{a[1]}] {Path(a[2]).name}", file=sys.stderr)
 
     print(f"Upgrade of {folder}  ->  library {dl_dir}", file=sys.stderr)
-    print("(real lossless added to the library; fake sources sent to trash)",
+    print(("(verified copies added; originals sent to trash after verification)"
+           if args.trash_original else "(verified copies added; originals retained)"),
           file=sys.stderr)
     outcomes = upgrade_mod.run_upgrade(
         folder, root=root, staging_dir=staging, download_dir=dl_dir,
         preset=preset, exclude_names=args.exclude,
         limit=args.limit, profile=args.profile, progress=progress, log_path=log_path,
+        trash_original=args.trash_original,
     )
 
     from collections import Counter
@@ -158,9 +160,9 @@ def progress(*a) -> None:
     for action, n in counts.most_common():
         print(f"  {action:<16} {n:>5}")
 
-    replaced = [o for o in outcomes if o.action in (upgrade_mod.ACT_REPLACED, upgrade_mod.ACT_WOULD_REPLACE)]
+    replaced = [o for o in outcomes if o.action in (upgrade_mod.ACT_REPLACED, upgrade_mod.ACT_KEPT_BESIDE, upgrade_mod.ACT_WOULD_REPLACE)]
     if replaced:
-        print(f"\n  --- {len(replaced)} AUTHENTIC upgrade(s) ---")
+        print(f"\n  --- {len(replaced)} accepted upgrade(s) ---")
         for o in replaced:
             print(f"  {o.artist} - {o.title}  cutoff {o.new_cutoff_hz:.0f} Hz")
 
@@ -601,7 +603,7 @@ def progress(*a) -> None:
         print(f"  {action:<16} {n:>5}")
     acquired = [o for o in outcomes if o.action == upgrade_mod.ACT_ACQUIRED]
     if acquired:
-        print(f"\n  --- {len(acquired)} AUTHENTIC track(s) in the library ---")
+        print(f"\n  --- {len(acquired)} accepted track(s) in the library ---")
         for o in acquired:
             print(f"  {o.artist} - {o.title}  cutoff {o.new_cutoff_hz:.0f} Hz")
     buy_html = stores_mod.write_unfindable(outcomes, paths.outputs_dir(), src.stem)
@@ -625,9 +627,10 @@ def progress(i: int, total: int, f: Path) -> None:
     print(f"Import of {src}  ->  library {dl_dir}", file=sys.stderr)
     stats = upgrade_mod.import_folder(src, dl_dir, exclude_names=args.exclude, progress=progress)
     print(f"\n=== Import: {src.name} ===")
-    print(f"  {stats['kept']:>5}  real lossless moved to the library")
-    print(f"  {stats['duplicates']:>5}  duplicates (already in library) -> trash")
-    print(f"  {stats['trashed']:>5}  non-lossless -> trash")
+    print(f"  {stats['kept']:>5}  accepted files moved to the library")
+    print(f"  {stats['duplicates']:>5}  confirmed duplicates retained at source")
+    print(f"  {stats['retained']:>5}  below threshold, retained for review")
+    print(f"  {stats['errors']:>5}  unreadable / failed, retained")
     print(f"  {stats['total']:>5}  files scanned in total")
     return 0
 
@@ -715,6 +718,8 @@ def build_parser() -> argparse.ArgumentParser:
     p_up.add_argument("--profile", default=None,
                       help="force a sldl profile (default: derived from --preset)")
     p_up.add_argument("-v", "--verbose", action="store_true", help="show each scanned file")
+    p_up.add_argument("--trash-original", action="store_true",
+                      help="send originals to trash only after verified installation (default: retain)")
     p_up.set_defaults(func=_cmd_upgrade)
 
     p_sc = sub.add_parser("scrape", help="scrape favorites/tracklists -> CSV want-list (-> acquire)")
@@ -743,7 +748,7 @@ def build_parser() -> argparse.ArgumentParser:
     p_ac.set_defaults(func=_cmd_acquire)
 
     p_im = sub.add_parser("import",
-                          help="migrate a folder into the library (lossless kept, the rest trashed)")
+                          help="move accepted files into the library; retain rejects, errors and duplicates")
     p_im.add_argument("folder", help="folder to import/sort")
     p_im.add_argument("--download-dir", help="target library (default: config / ~/Music/DDD)")
     p_im.add_argument("-x", "--exclude", action="append", default=[], metavar="NAME",

ddd/core/content.py

@@ -0,0 +1,35 @@
+"""Content checks shared by scan, rename and library imports."""
+
+from collections import defaultdict
+import hashlib
+from pathlib import Path
+
+
+def file_hash(path):
+    digest = hashlib.sha256()
+    with open(path, "rb") as stream:
+        for block in iter(lambda: stream.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def duplicate_paths(files):
+    """Group readable files by size then SHA-256; unreadable files stay unique."""
+    sizes = defaultdict(list)
+    for path in dict.fromkeys(Path(p) for p in files):
+        try:
+            sizes[path.stat().st_size].append(path)
+        except OSError:
+            continue
+    groups = []
+    for candidates in sizes.values():
+        if len(candidates) < 2:
+            continue
+        hashes = defaultdict(list)
+        for path in candidates:
+            try:
+                hashes[file_hash(path)].append(path)
+            except OSError:
+                continue
+        groups.extend(group for group in hashes.values() if len(group) > 1)
+    return groups

ddd/core/detect.py

@@ -122,8 +122,8 @@ def forensic_classify(
     # aucune signature MP3 confirmee (est1 None) + pas de red flag conteneur. Un vrai 320
     # plafonne ~20.5 kHz (< 95% Nyquist), donc il ne remonte PAS ici -> promotion sure.
     if base_v != q.LOSSLESS and est1 is None and near_nyquist and not flac_red_flag:
-        reason = f"cutoff {cutoff:.0f} Hz >= 95% Nyquist, aucune signature MP3 (Rule 1/8) -> authentique"
-        return q.LOSSLESS, "high", reason, est, signals
+        reason = f"cutoff {cutoff:.0f} Hz >= 95% Nyquist, bande passante large (Rule 1/8)"
+        return q.LOSSLESS, "uncertain", reason + "; compression history unknown", est, signals
 
     # ZONE GRISE : le legacy retrograde (HQ/DOUTEUX) mais Rule 1 ne confirme PAS de MP3.
     # On ne change NI le verdict NI le cutoff (is_accepted inchange) : juste la confiance.

ddd/core/fsutil.py

@@ -8,8 +8,44 @@
 from __future__ import annotations
 
 import shutil
+import os
 from pathlib import Path
 
+from .content import file_hash
+
+
+def verified_deposit(src, dest_dir) -> Path:
+    """Copy into an exclusively reserved name and verify bytes before removing staging.
+
+    On failure only our incomplete copy is removed. The source remains available.
+    A collision keeps both files; existing library files are never overwritten.
+    """
+    src, dest_dir = Path(src), Path(dest_dir)
+    dest_dir.mkdir(parents=True, exist_ok=True)
+    dest = dest_dir / src.name
+    if dest.resolve() == src.resolve():
+        raise ValueError("download staging must be separate from the library")
+    index = 0
+    while True:
+        try:
+            output = dest.open("xb")
+            break
+        except FileExistsError:
+            index += 1
+            dest = dest_dir / f"{src.stem} ({index}){src.suffix}"
+    try:
+        with output, src.open("rb") as source:
+            shutil.copyfileobj(source, output)
+            output.flush()
+            os.fsync(output.fileno())
+        if src.stat().st_size != dest.stat().st_size or file_hash(src) != file_hash(dest):
+            raise OSError("download copy verification failed")
+    except BaseException:
+        dest.unlink(missing_ok=True)
+        raise
+    src.unlink()
+    return dest
+
 
 def safe_move(src, dest_dir, *, dry_run: bool = False) -> Path:
     """Deplace `src` dans `dest_dir`, en suffixant ' (n)' si le nom est deja pris.