diff --git a/.changeset/calm-spoons-copy.md b/.changeset/calm-spoons-copy.md new file mode 100644 index 0000000..e81a685 --- /dev/null +++ b/.changeset/calm-spoons-copy.md @@ -0,0 +1,5 @@ +--- +'vlurp': minor +--- + +Add `vlurp SOURCE... DEST` copying with `pkg:github` PURL sources and glob selection inside `#subpath`. Copies now support familiar basename semantics, multiple remote sources, embedded refs, destination-relative lineage, vlurpfile pin/upgrade integration, and traversal-safe preflight planning. Repeatable `--glob` and case-insensitive `--iglob` patterns filter files beneath cp sources, and existing presets compose with the new interface. Repository shorthand, presets, and `--filter` remain fully supported. diff --git a/README.md b/README.md index 0f6586f..d59e04e 100644 --- a/README.md +++ b/README.md @@ -17,42 +17,86 @@ npm install -g vlurp or run directly: ```sh -npx vlurp / +npx vlurp mattpocock/skills --preset skills ``` ## Quick start -Fetch a repo's Claude config: +The easy case is repository shorthand. Fetch the useful skill files from a repository with a maintained preset: +```sh +vlurp mattpocock/skills --preset skills +``` + +This writes the selected repository content beneath `./mattpocock/skills`. Pin it to a commit when you want a reproducible fetch: + +```sh +vlurp mattpocock/skills --preset skills --ref 2ab9580 +``` + +Use `-d` to choose an output root, `--as` to give the fetched directory a specific name, or `--filter` for repository-relative patterns: + +```sh +vlurp mattpocock/skills -d ./vendor --preset skills +vlurp mattpocock/skills -d ./skills --as writing --filter 'skills/in-progress/writing-*' ``` -$ vlurp eyaltoledano/claude-task-master -d ./vlurp - eyaltoledano/claude-task-master@HEAD - 2 files - ./vlurp/eyaltoledano/claude-task-master/ +When you need exact remote sources and `cp` placement, use PURL operands. This copies matching directories directly into `./skills/`: + ``` +$ vlurp 'pkg:github/mattpocock/skills#skills/in-progress/writing-*' ./skills/ -Fetch skill files pinned to a commit, flattened into a named directory: + ./skills/writing-beats/ + ./skills/writing-fragments/ + ./skills/writing-shape/ +``` +Embed the ref in the PURL to pin the same copy: + +``` +$ vlurp 'pkg:github/mattpocock/skills@2ab9580#skills/in-progress/writing-*' ./skills/ ``` -$ vlurp obra/superpowers -d .claude/skills --preset skills --ref e4f5a6b - obra/superpowers@e4f5a6b - 22 files (preset: skills) - .claude/skills/obra/superpowers/ +The command follows familiar `cp SOURCE... DEST` rules. A selected directory or file is copied by basename into an existing destination directory. If a single source is copied to a missing destination, that destination becomes the copy. + +Copy multiple exact sources: + +```sh +vlurp \ + 'pkg:github/anthropics/skills@main#skills/pdf' \ + 'pkg:github/anthropics/skills@main#skills/slides' \ + ./skills/ ``` -Fetch a specific directory from deep inside a repo: +## Presets + +Presets are a first-class way to fetch common repository structures without spelling out globs. They work with both repository shorthand and the cp-style interface: +```sh +vlurp obra/superpowers --preset skills -d ./.claude/skills --ref e4f5a6b +vlurp eyaltoledano/claude-task-master --preset claude -d ./config +vlurp 'pkg:github/obra/superpowers' ./.claude/skills --preset skills ``` -$ vlurp whilp/dotfiles -d .claude/skills \ - --filter ".claude/skills/duckdb-json/**" --as duckdb - whilp/dotfiles@HEAD - 3 files - .claude/skills/duckdb/ +| Preset | Selects | +|--------|---------| +| `claude` | `.claude/**`, `CLAUDE.md` | +| `skills` | `skills/**`, `SKILL.md`, Markdown support files | +| `agents` | `agents/**`, `commands/**`, Markdown support files | +| `docs` | Markdown documentation excluding boilerplate | +| `all-md` | All Markdown files | +| `minimal` | Only `.claude/**` and `CLAUDE.md` | + +With cp-style operands, a preset filters the files beneath each selected source without changing where that source is placed. Add `--glob` or `--iglob` to refine a preset; explicit patterns are applied after the preset. + +```sh +vlurp 'pkg:github/obra/superpowers#skills' ./skills \ + --preset skills \ + --glob '!**/experimental/**' ``` +`--glob` is case-sensitive and `--iglob` is case-insensitive. Both are repeatable. A leading `!` excludes matches. If any positive pattern is present, unmatched files are excluded; with exclusion-only patterns, unmatched files remain included. + Check that nothing has been modified since you fetched: ``` @@ -109,17 +153,13 @@ Process multiple repos from a `.vlurpfile`: # .vlurpfile # Official Anthropic skills -vlurp anthropics/skills -d ./vlurp --filter "skills/**" --filter "template/**" +vlurp 'pkg:github/anthropics/skills@b7c8d9e#skills/*' ./skills/ # obra/superpowers -- Core agent patterns -vlurp obra/superpowers -d ./vlurp --filter "skills/**" --filter ".claude/**" - -# DuckDB skills from assorted dotfiles -vlurp whilp/dotfiles -d ./vlurp --filter ".claude/skills/duckdb-json/**" -vlurp PovertyAction/ipa-research-data-science-hub -d ./vlurp --filter ".claude/skills/duckdb/**" +vlurp obra/superpowers --preset skills -d ./skills --ref e4f5a6b # Microsoft Amplifier -- multi-agent framework -vlurp microsoft/amplifier -d ./vlurp --filter "**/*.md" +vlurp 'pkg:github/microsoft/amplifier@4a5b6c7#**/*.md' ./docs/ ``` ``` @@ -129,7 +169,10 @@ $ vlurp batch .vlurpfile ## Commands ``` -vlurp Fetch from a GitHub repo or gist +vlurp REPOSITORY [options] Fetch a repository using shorthand +vlurp REPOSITORY --preset NAME Fetch using a named selection preset +vlurp SOURCE... DEST Copy precise PURL sources into local files +vlurp SOURCE... DEST --preset NAME Copy PURL sources using a preset vlurp batch Process a .vlurpfile (batch fetch) vlurp verify Check file integrity against lineage vlurp pin [source] Pin sources to current upstream HEAD @@ -144,34 +187,27 @@ vlurp catalog-diff [old] [new] Compare catalog snapshots ## Flags ``` --d Root output directory ---ref Pin to a git ref (commit, tag, branch) ---as Flatten output into named directory ---preset Use a preset filter set ---filter Glob pattern for file matching (repeatable) ---auto Auto-detect repo structure +-d Root output directory for repository selection mode +--ref Pin a repository-mode fetch to a Git ref +--as Override its output directory name +--preset Apply a maintained named selection preset +--glob Apply a case-sensitive cp transfer pattern +--iglob Apply a case-insensitive cp transfer pattern +--filter Add repository-mode selection patterns +--auto Detect a suitable repository preset --dry-run, -n Preview without writing --force, -f Overwrite without prompting --json Machine-readable output (catalog-diff) --vlurpfile Explicit .vlurpfile path (upgrade) ``` -## Presets - -``` -claude .claude/**, CLAUDE.md -skills skills/**, SKILL.md, **/*.md -agents agents/**, commands/**, **/*.md -docs **/*.md (excluding boilerplate) -all-md **/*.md -minimal .claude/**, CLAUDE.md only -``` +PURL operands determine remote sources and local placement. `--preset`, `--glob`, and `--iglob` determine the files transferred beneath those sources. Repository shorthand with `--preset` or `--filter` remains supported. ## Feature guides | Guide | Covers | |-------|--------| -| **[Fetching & Filtering](doc/fetch.md)** | Sources, globs, presets, `--ref`, `--as`, `--auto` | +| **[Fetching and Copying](doc/fetch.md)** | Repository shorthand, presets, PURL sources, refs, and cp semantics | | **[The .vlurpfile](doc/vlurpfile.md)** | Batch processing, manifest format, intent vs reality | | **[Supply Chain Security](doc/supply-chain.md)** | Lineage, verify, pin, scan, threat model | | **[Upgrades & Change Detection](doc/upgrade.md)** | outdated, diff, upgrade, catalog, catalog-diff | diff --git a/doc/fetch.md b/doc/fetch.md index dfa3cec..23dbe82 100644 --- a/doc/fetch.md +++ b/doc/fetch.md @@ -1,127 +1,156 @@ -# Fetching & Filtering +# Fetching and Copying from GitHub -vlurp fetches files from GitHub repositories and gists. It downloads tarballs -- not git clones -- so you get files without history, without `.git` directories, and without executing anything on your machine. +The easy case is repository shorthand: -You do not need a forty-nine step installer to get a text file onto your disk. - -## Sources +```sh +vlurp mattpocock/skills +vlurp mattpocock/skills --preset skills +``` -vlurp accepts three source formats: +Repository shorthand accepts GitHub URLs too. It writes beneath `.//` by default and preserves repository-relative paths. ```sh -# user/repo shorthand -vlurp obra/superpowers +# Choose a maintained selection policy +vlurp mattpocock/skills --preset skills -# Full GitHub URL -vlurp https://github.com/microsoft/amplifier +# Pin the fetch and choose its output root +vlurp mattpocock/skills --preset skills --ref 2ab9580 -d ./vendor -# Gist URL -vlurp https://gist.github.com/user/abc123def456 +# Name the destination and provide repository-relative patterns +vlurp mattpocock/skills -d ./skills --as writing \ + --filter 'skills/in-progress/writing-*' + +# Let vlurp detect a suitable repository structure +vlurp owner/repository --auto ``` -All three produce the same result: files on disk. +Use `--preset` for a named, maintained selection policy; `--filter` for one or more explicit repository-relative patterns; `--ref` to pin a commit, tag, or branch; `-d` to choose an output root; and `--as` to name the fetched directory. -## Output directory +## Precise copy operands -By default, vlurp writes to `.//`. Use `-d` to set a root directory: +When you need exact remote source paths and `cp` destination placement, treat the GitHub repository as a remote filesystem: ```sh -vlurp obra/superpowers -d .claude/skills -# writes to .claude/skills/obra/superpowers/ +vlurp SOURCE... DEST ``` -Use `--as` to flatten the path: +The sources are Package URLs (PURLs). The destination is a local file or directory. This form complements repository shorthand; it does not replace it. -```sh -vlurp obra/superpowers -d .claude/skills --as superpowers -# writes to .claude/skills/superpowers/ +## Source grammar + +```text +pkg:github//@# ``` -`--as` strips the `owner/repo` prefix and all internal path nesting. When you're fetching skill files from deep inside a repo's directory tree, `--as` puts them where you can find them. +The ref and subpath are optional: ```sh -vlurp whilp/dotfiles -d ./skills --filter ".claude/skills/duckdb-json/**" --as duckdb -# instead of: ./skills/whilp/dotfiles/.claude/skills/duckdb-json/SKILL.md -# you get: ./skills/duckdb/SKILL.md +# Repository root at its default branch +vlurp 'pkg:github/obra/superpowers' ./superpowers + +# Exact directory at a tag, branch, or commit +vlurp 'pkg:github/anthropics/skills@b7c8d9e#skills/pdf' ./skills/ + +# Exact file +vlurp 'pkg:github/user/repo@v1.0.0#docs/guide.md' ./docs/ ``` -## Filtering +When `@ref` is omitted, GitHub's default branch is resolved at execution time. A commit SHA is the reproducible choice. -By default, vlurp matches a broad set of agent-relevant files: `.claude/**`, `CLAUDE.md`, `**/*.md`, `agents/**`, `commands/**`. Use `--filter` to override: +## PURL patterns + +A canonical PURL identifies one exact repository subpath. vlurp extends the `#subpath` with glob selection: ```sh -# Only TypeScript files -vlurp user/repo --filter "*.ts" --filter "*.tsx" +vlurp 'pkg:github/mattpocock/skills#skills/in-progress/writing-*' ./skills/ +``` -# Only a specific directory -vlurp user/repo --filter "lib/**" +This produces: -# Combine inclusions and exclusions -vlurp user/repo --filter "**/*.md" --filter "!README.md" +```text +./skills/writing-beats/ +./skills/writing-fragments/ +./skills/writing-shape/ ``` -`--filter` accepts glob patterns. Prefix with `!` to exclude. Multiple `--filter` flags are combined. +Only the subpath can contain glob syntax. The provider, owner, repository, and ref must be exact. Quote patterned PURLs so your shell does not interpret `*`, `?`, or brackets. -## Presets +Every exact `pkg:github` PURL is a valid vlurp source. Once unescaped glob syntax appears in its subpath, the operand is a vlurp source pattern rather than a canonical PURL identity. -Presets are named filter sets for common repo structures: +## Transfer patterns and presets -``` -claude .claude/**, CLAUDE.md -skills skills/**, SKILL.md, **/*.md -agents agents/**, commands/**, **/*.md -docs **/*.md (excluding boilerplate) -all-md **/*.md -minimal .claude/**, CLAUDE.md only -``` +The source operands determine what is copied and the destination determines where it is placed. Repeatable transfer patterns can limit the files beneath each selected source: ```sh -vlurp obra/superpowers --preset skills -vlurp eyaltoledano/claude-task-master --preset claude +vlurp 'pkg:github/user/repo#docs' ./docs/ --glob '**/*.md' --glob '!README.md' +vlurp 'pkg:github/user/repo#skills' ./skills/ --iglob '**/skill.md' ``` -Presets exclude boilerplate by default (README.md, LICENSE, CONTRIBUTING.md, etc.). If you want everything, use `all-md` or explicit `--filter` globs. +- `--glob PATTERN` is case-sensitive. +- `--iglob PATTERN` is case-insensitive. +- Prefix a pattern with `!` to exclude matching files. +- If there is any positive pattern, unmatched files are excluded. If every pattern is negative, unmatched files are included. +- Rules from a preset are applied first, then `--glob` rules, then `--iglob` rules. A later matching rule wins. + +Patterns are matched against paths relative to each selected source. They change the transfer set, not cp destination placement. -## Auto-detect +Presets are maintained, named collections of these transfer rules for common repository structures: -`--auto` inspects the tarball contents and picks the best preset: +| Preset | Selects | +|--------|---------| +| `claude` | `.claude/**`, `CLAUDE.md` | +| `skills` | `skills/**`, `SKILL.md`, Markdown support files | +| `agents` | `agents/**`, `commands/**`, Markdown support files | +| `docs` | Markdown documentation excluding boilerplate | +| `all-md` | All Markdown files | +| `minimal` | Only `.claude/**` and `CLAUDE.md` | ```sh -vlurp microsoft/amplifier --auto +vlurp obra/superpowers --preset skills -d ./.claude/skills --ref e4f5a6b +vlurp eyaltoledano/claude-task-master --preset claude -d ./config +vlurp 'pkg:github/obra/superpowers' ./.claude/skills --preset skills +vlurp 'pkg:github/obra/superpowers#skills' ./skills --preset skills --glob '!**/drafts/**' ``` -If the repo has a `.claude/` directory, `--auto` uses `claude`. If it has `skills/` or `SKILL.md` files, it uses `skills`. Otherwise it falls back to the default filter set. +The first two commands use repository shorthand, which preserves repository-relative layout; `--ref`, `-d`, `--as`, and `--auto` are supported there. The latter two use cp placement. Presets are first-class in both forms. + +## Copy semantics + +vlurp follows recursive `cp` operand semantics: -## Pinning with `--ref` +- A source copied into an existing directory is placed beneath it by basename. +- A single source copied to a missing destination creates that destination. +- Multiple sources require an existing destination directory. +- Two selected sources with the same basename are rejected before anything is written. +- Existing matching files are overwritten; unrelated files remain in place. +- Without `--force`, vlurp asks before copying over an existing destination path. -`--ref` pins a fetch to a specific git commit, tag, or branch: +Examples: ```sh -vlurp obra/superpowers --ref e4f5a6b -vlurp anthropics/skills --ref v1.2.0 -vlurp microsoft/amplifier --ref main +# Rename one exact file +vlurp 'pkg:github/user/repo#docs/guide.md' ./README.md + +# Copy two directories into an existing directory +vlurp \ + 'pkg:github/anthropics/skills@main#skills/pdf' \ + 'pkg:github/anthropics/skills@main#skills/slides' \ + ./skills/ ``` -When `--ref` is a commit SHA, the fetch is immutable. The same SHA always produces the same content. This is the foundation of everything in vlurp's security model. +## Safety -When `--ref` is omitted, vlurp fetches the default branch HEAD. This is the mutable case. Whatever the author pushed most recently is what you get. If the author pushed a malicious change thirty seconds ago, you have it. Pin your refs. +Subpaths must be relative and cannot contain empty, `.` or `..` segments or encoded path separators. Archive paths that escape the repository are discarded, and symbolic and hard links are not extracted. Copy plans are validated for destination collisions before writing. -## Dry run - -`--dry-run` shows what would be fetched without writing anything: +## Preview and overwrite ```sh -$ vlurp obra/superpowers --preset skills --dry-run - - obra/superpowers@HEAD - would fetch 22 files (preset: skills) - would write to ./obra/superpowers/ +vlurp 'pkg:github/user/repo#skills/*' ./skills/ --dry-run +vlurp 'pkg:github/user/repo#skills/*' ./skills/ --force ``` -## Lineage +`--dry-run` resolves the remote selector and prints the concrete copy plan without writing files or lineage. -Every fetch produces a lineage record in `.vlurp.jsonl` with SHA-256 hashes of every file. This happens automatically. See [Supply Chain Security](supply-chain.md) for details. - -## Scanning +## Lineage -Fetched content is scanned for prompt injection, tool escalation, and exfiltration patterns by default. See [Supply Chain Security](supply-chain.md) for details. +Every completed copy writes `.vlurp.jsonl` beside the copied set. Its file keys are relative to the lineage file, so `vlurp verify ` can check the exact destination layout without reconstructing it from the repository name. diff --git a/doc/supply-chain.md b/doc/supply-chain.md index aa0f845..043765b 100644 --- a/doc/supply-chain.md +++ b/doc/supply-chain.md @@ -21,13 +21,13 @@ When a skill file gets injected into an LLM context, these attacks are possible: ## Pin your refs -`--ref` pins a fetch to a specific git commit SHA. Pinned content is immutable. The same SHA always produces the same tarball. +`@ref` in a PURL source pins a fetch to a specific git commit SHA. Pinned content is immutable. The same SHA always produces the same tarball. ```sh -vlurp obra/superpowers -d .claude/skills --preset skills --ref e4f5a6b +vlurp 'pkg:github/obra/superpowers@e4f5a6b#skills/*' .claude/skills/ ``` -When `--ref` is omitted, vlurp fetches whatever is on the default branch right now. If the author pushed a malicious change thirty seconds ago, you have it. This is how supply chain attacks work: the content is fine when you first look, and different when you fetch. +When `@ref` is omitted, vlurp fetches whatever is on the default branch right now. If the author pushed a malicious change thirty seconds ago, you have it. This is how supply chain attacks work: the content is fine when you first look, and different when you fetch. To pin all unpinned sources in a `.vlurpfile` to the current upstream HEAD: @@ -43,13 +43,12 @@ Every `vlurp fetch` or `vlurp batch` produces a lineage record in `.vlurp.jsonl` ```json { - "source": "github:obra/superpowers", + "source": "pkg:github/obra/superpowers@e4f5a6b#skills/*", "ref": "e4f5a6b7c8d9", "ref_type": "commit", "fetched_at": "2026-03-15T06:00:00Z", - "filters": [], - "preset": "skills", - "as": null, + "destination": ".", + "files_root_relative": true, "files": { "skills/tdd/SKILL.md": { "sha256": "e3b0c442...", "size": 3421 }, "skills/verify/SKILL.md": { "sha256": "7f83b165...", "size": 2891 } @@ -151,7 +150,7 @@ See [ATTEST.H2H.md](../ATTEST.H2H.md) for the current attestation design. The security model is four layers deep: ``` -PIN Immutable content via --ref +PIN Immutable content via @ref in the source PURL HASH SHA-256 of every file in .vlurp.jsonl VERIFY Check disk against lineage at any time SCAN Know what the content does before you inject it diff --git a/doc/upgrade.md b/doc/upgrade.md index 6588c32..f0aaae5 100644 --- a/doc/upgrade.md +++ b/doc/upgrade.md @@ -60,7 +60,7 @@ This is `terraform plan` for your agent context. You see what would change befor vlurp upgrade # Upgrade a specific source -vlurp upgrade obra/superpowers +vlurp upgrade github:obra/superpowers # Preview without modifying anything vlurp upgrade --dry-run @@ -76,9 +76,11 @@ vlurp upgrade --vlurpfile .vlurpfile.skills 3. **Compares** against pinned refs to find outdated sources 4. **Fetches** new content through the standard pipeline (download, extract, filter, hash, scan) 5. **Catalogs** new content and diffs against the previous catalog -6. **Rewrites** `.vlurpfile` with updated `--ref` values +6. **Rewrites** `.vlurpfile` with updated PURL `@ref` values 7. **Outputs** upgrade summary with catalog diff +For cp-style entries, repository refs live inside each PURL source. Upgrade groups selectors by repository, fetches the new commit through the same copy planner, and rewrites `@ref` without changing `#subpath`, quoting, or the destination. + The fetch pipeline is the same one used by `vlurp fetch` and `vlurp batch`. Upgraded content gets hashed, lineage gets written, scans run. No shortcuts. ### Dry run @@ -109,26 +111,26 @@ $ vlurp upgrade --dry-run ### Vlurpfile rewriting -`vlurp upgrade` rewrites the `.vlurpfile` to update `--ref` values. If a source had no `--ref`, one is added. Comments, blank lines, and argument ordering are preserved. +`vlurp upgrade` rewrites the `.vlurpfile` to update PURL `@ref` values. If a source had no ref, one is embedded. Comments, blank lines, quoting, subpaths, and destinations are preserved. Legacy entries continue to receive `--ref` during migration. -Before: +Before (cp-style): ```sh # Core agent patterns -vlurp obra/superpowers -d .claude/skills --preset skills --ref e4f5a6b +vlurp 'pkg:github/obra/superpowers@e4f5a6b#skills/*' .claude/skills/ # Multi-agent framework -vlurp microsoft/amplifier -d .claude/skills --filter "**/*.md" +vlurp 'pkg:github/microsoft/amplifier#**/*.md' .claude/docs/ ``` After `vlurp upgrade`: ```sh # Core agent patterns -vlurp obra/superpowers -d .claude/skills --preset skills --ref 9c8b7a6 +vlurp 'pkg:github/obra/superpowers@9c8b7a6#skills/*' .claude/skills/ # Multi-agent framework -vlurp microsoft/amplifier -d .claude/skills --filter "**/*.md" --ref d3e2f1a +vlurp 'pkg:github/microsoft/amplifier@d3e2f1a#**/*.md' .claude/docs/ ``` The superpowers entry's ref was updated. The amplifier entry gained a ref -- it was previously unpinned, now pinned to the version that was fetched. diff --git a/doc/vlurpfile.md b/doc/vlurpfile.md index c5ad9bb..83efbb7 100644 --- a/doc/vlurpfile.md +++ b/doc/vlurpfile.md @@ -1,123 +1,74 @@ # The .vlurpfile -A `.vlurpfile` is a manifest of fetch commands. One vlurp invocation per line, comments with `#`. It is a shell script you can also run manually. It is human-readable, human-editable, and greppable. +A `.vlurpfile` is a manifest of copy commands. Each non-comment line is a complete vlurp invocation that can also be pasted into a terminal. -It is not a `package.json`. There is no dependency resolution. There are no lifecycle hooks. There is no semver. Each line is an independent fetch operation. +It is not a package manifest: there is no dependency resolution, registry, semver solving, or lifecycle execution. ## Format ```sh -# .vlurpfile -- Agent skill sources, reviewed and pinned +# .vlurpfile -- reviewed and pinned remote files -# obra/superpowers -- Core agent patterns -vlurp obra/superpowers -d .claude/skills --preset skills --ref e4f5a6b - -# whilp/dotfiles -- DuckDB skills -vlurp whilp/dotfiles -d .claude/skills --filter ".claude/skills/duckdb-json/**" --as duckdb --ref 6fc9349 - -# eyaltoledano/claude-task-master -- Task management -vlurp eyaltoledano/claude-task-master -d .claude/skills --preset claude --ref 29e67fa +vlurp 'pkg:github/anthropics/skills@b7c8d9e#skills/*' ./.claude/skills/ +vlurp 'pkg:github/obra/superpowers@e4f5a6b#skills' ./skills/ --preset skills --glob '!**/drafts/**' +vlurp obra/superpowers --preset skills -d ./.claude/skills --ref e4f5a6b +vlurp 'pkg:github/whilp/dotfiles@6fc9349#.claude/skills/duckdb-json' ./skills/ ``` Rules: -- Lines starting with `#` are comments -- Blank lines are ignored -- Each command line is a complete `vlurp` invocation -- Arguments are parsed the same way as the CLI -- You can run any line by itself: copy it, paste it into your terminal +- Lines starting with `#` and blank lines are ignored. +- PURL copy entries have `SOURCE... DEST` operands followed by repeatable `--glob` and `--iglob` patterns or `--preset`. +- Repository entries may use named presets, filters, and their existing output options. +- Both single and double quotes are supported. +- Patterned PURLs should be quoted for parity with commands pasted into a shell. +- `--force`, `--quiet`, and `--dry-run` are valid invocation flags. +- Presets work in both PURL copy and repository shorthand entries. ## Batch processing -`vlurp batch` processes an entire `.vlurpfile`: - ```sh vlurp batch .vlurpfile -``` - -Each line executes in sequence. Progress is reported per-source. If a source fails, vlurp continues with the remaining sources and reports failures at the end. - -Preview what would happen without writing anything: - -```sh vlurp batch .vlurpfile --dry-run -``` - -Force overwrite existing content: - -```sh vlurp batch .vlurpfile --force ``` -## File naming conventions - -You can name your vlurpfile anything. Common conventions: +Entries run sequentially. A failed entry is reported without preventing later entries from running. -``` -.vlurpfile Default -.vlurpfile.skills Skills only -.vlurpfile.claude Claude Code configuration -.vlurpfile.team Shared team configuration -``` +## Intent and reality -vlurp looks for `.vlurpfile` by default when running `vlurp upgrade` or `vlurp pin` without an explicit path. +The `.vlurpfile` records intent: remote identities, selected paths, refs, and local destinations. `.vlurp.jsonl` records reality: the source operand, fetch time, destination-relative paths, and hashes of the bytes written. -## Intent vs reality +Commit both. Review the `.vlurpfile` for what should be copied and lineage for what was copied. -The `.vlurpfile` records **intent**: what you want to fetch, from where, with what filters and pins. +## Pin and upgrade -The `.vlurp.jsonl` records **reality**: what was actually fetched, when, with what SHA-256 hashes. +`vlurp pin` resolves the default-branch HEAD for each unpinned PURL and embeds the short commit in `@ref`: +```diff +-vlurp 'pkg:github/obra/superpowers#skills/*' ./skills/ ++vlurp 'pkg:github/obra/superpowers@e4f5a6b#skills/*' ./skills/ ``` -.vlurpfile "fetch obra/superpowers at ref e4f5a6b with preset skills" -.vlurp.jsonl "fetched obra/superpowers at e4f5a6b on 2026-03-15, got 22 files, here are the hashes" -``` - -Both files are committed to git. Both are reviewed in PRs. The `.vlurpfile` is reviewed for intent ("should we add this source?"). The `.vlurp.jsonl` is reviewed for integrity ("did the fetch produce what we expected?"). They answer different questions. - -Some tools use a lock file that stores a hash from the GitHub Trees API -- not a hash of the content that was actually written to disk. That is a receipt for what the server said, not proof of what you have. vlurp hashes the files after extraction. The `.vlurp.jsonl` describes reality. - -## Editing - -You can edit a `.vlurpfile` with any text editor. It's a text file. -To pin all unpinned sources to the current upstream HEAD: +`vlurp upgrade` compares embedded refs with upstream HEAD, reruns the same copy plan, and updates each affected PURL in place. Comments, whitespace, quoting, destinations, and subpaths are preserved. ```sh vlurp pin -``` - -To upgrade all sources to the latest upstream and update their `--ref` values: - -```sh +vlurp outdated .vlurpfile +vlurp upgrade --dry-run vlurp upgrade ``` -Both commands rewrite the `.vlurpfile` in place, preserving comments, blank lines, and argument ordering. See [Upgrades & Change Detection](upgrade.md) for details. +When one command contains sources from multiple repositories, each source is pinned and upgraded independently. -## Example: real-world vlurpfile +## Recommended review workflow ```sh -# .vlurpfile -- Production agent skills -# Last reviewed: 2026-03-15 by @indexzero -# -# Review checklist: -# 1. vlurp batch .vlurpfile --dry-run -# 2. vlurp scan .claude/skills -# 3. git diff .vlurp.jsonl - -# Official Anthropic skills -vlurp anthropics/skills -d .claude/skills --filter "skills/**" --filter "template/**" --ref b7c8d9e - -# Core agent patterns (obra) -vlurp obra/superpowers -d .claude/skills --filter "skills/**" --filter ".claude/**" --ref e4f5a6b - -# DuckDB skills from assorted dotfiles -vlurp whilp/dotfiles -d .claude/skills --filter ".claude/skills/duckdb-json/**" --as duckdb --ref 6fc9349 -vlurp PovertyAction/ipa-research-data-science-hub -d .claude/skills --filter ".claude/skills/duckdb/**" --as duckdb-ipa - -# Microsoft Amplifier -- multi-agent framework -vlurp microsoft/amplifier -d .claude/skills --filter "**/*.md" --ref 4a5b6c7 +vlurp batch .vlurpfile --dry-run +vlurp batch .vlurpfile +vlurp verify ./skills +vlurp scan ./skills +git diff -- .vlurpfile skills/.vlurp.jsonl ``` -The comment block at the top is a review checklist. When this file changes in a PR, the reviewer runs those three commands. The `.vlurpfile` is the table of contents. The `.vlurp.jsonl` diff in the same PR is the proof that the content matches. +The dry run shows concrete remote matches and destination paths. Verification checks the resulting bytes; scanning describes the instruction and tool surface. diff --git a/src/catalog.js b/src/catalog.js index a7b1a30..659e64b 100644 --- a/src/catalog.js +++ b/src/catalog.js @@ -18,7 +18,7 @@ export async function buildCatalog(resolvedPath) { for (const record of records) { const sourceId = record.source.replace(/^github:/, ''); - const prefix = record.as || sourceId; + const prefix = record.files_root_relative ? '' : record.as || sourceId; for (const filePath of Object.keys(record.files || {})) { if (basename(filePath) !== 'SKILL.md') { @@ -42,10 +42,10 @@ export async function buildCatalog(resolvedPath) { } async function buildSkillEntry(resolvedPath, record, prefix, { filePath, scanSummary }) { - const skillDir = - filePath === 'SKILL.md' ? prefix : join(prefix, filePath.replace(/\/SKILL\.md$/, '')); + const relativeSkillPath = prefix ? join(prefix, filePath) : filePath; + const skillDir = relativeSkillPath.replace(/\/?SKILL\.md$/, ''); const skillName = basename(skillDir); - const fullSkillPath = join(resolvedPath, prefix, filePath); + const fullSkillPath = join(resolvedPath, relativeSkillPath); let description = ''; let frontmatter = null; @@ -62,7 +62,7 @@ async function buildSkillEntry(resolvedPath, record, prefix, { filePath, scanSum .filter(f => f !== filePath && f.startsWith(filePath.replace(/SKILL\.md$/, ''))) .map(f => basename(f)); - const scanKey = join(prefix, filePath); + const scanKey = relativeSkillPath; const fileScan = scanSummary.details[scanKey] || {}; return { @@ -70,7 +70,7 @@ async function buildSkillEntry(resolvedPath, record, prefix, { filePath, scanSum data: { source: record.source, ref: record.ref, - path: join(prefix, filePath), + path: relativeSkillPath, name: frontmatter?.name || skillName, version: frontmatter?.version || null, description, diff --git a/src/cli.js b/src/cli.js index fd59c40..362c612 100644 --- a/src/cli.js +++ b/src/cli.js @@ -5,6 +5,7 @@ import React from 'react'; import { BatchCommand } from './commands/batch.js'; import { CatalogCommand } from './commands/catalog.js'; import { CatalogDiffCommand } from './commands/catalog-diff.js'; +import { CopyCommand } from './commands/copy.js'; import { DiffCommand } from './commands/diff.js'; import { FetchCommand } from './commands/fetch.js'; import { OutdatedCommand } from './commands/outdated.js'; @@ -14,8 +15,23 @@ import { UpgradeCommand } from './commands/upgrade.js'; import { VerifyCommand } from './commands/verify.js'; import { PRESETS } from './presets.js'; +const DEFAULT_FILTERS = [ + '.claude/**', + 'CLAUDE.md', + '*.md', + '**/*.md', + '!README.md', + '!CONTRIBUTING.md', + '!LICENSE.md', + '!CHANGELOG.md', + '!CODE_OF_CONDUCT.md', + 'agents/**', + 'commands/**' +]; + const j = jack({ - usage: 'vlurp [command] [source] [options]' + usage: + 'vlurp REPOSITORY [options]\n vlurp SOURCE... DEST [options]\n vlurp COMMAND [arguments] [options]' }) .description('A fun CLI tool to quickly fetch GitHub repositories and gists') .opt({ @@ -43,20 +59,13 @@ const j = jack({ }) .optList({ filter: { - description: 'Glob patterns to filter files (see defaults in help)', - default: [ - '.claude/**', - 'CLAUDE.md', - '*.md', - '**/*.md', - '!README.md', - '!CONTRIBUTING.md', - '!LICENSE.md', - '!CHANGELOG.md', - '!CODE_OF_CONDUCT.md', - 'agents/**', - 'commands/**' - ] + description: 'Glob patterns to filter files in repository selection mode' + }, + glob: { + description: 'Case-sensitive transfer pattern; prefix with ! to exclude' + }, + iglob: { + description: 'Case-insensitive transfer pattern; prefix with ! to exclude' } }) .flag({ @@ -91,7 +100,8 @@ if (values.help) { console.log(`vlurp - A fun CLI tool to quickly fetch GitHub repositories and gists Commands: - vlurp Fetch a single repository + vlurp [options] Fetch a repository using shorthand + vlurp ... Copy precise PURL sources with cp semantics vlurp batch Process a .vlurpfile vlurp upgrade [source] Upgrade outdated sources to upstream HEAD vlurp verify Verify file integrity against lineage records @@ -103,6 +113,17 @@ Commands: vlurp catalog-diff Compare two catalog snapshots Usage: + vlurp / Fetch useful text files + vlurp / --preset Fetch using a named preset + vlurp / --filter '' Fetch repository-relative matches + vlurp 'pkg:github//#' DEST Copy an exact remote path + vlurp 'pkg:github//#' DEST Select remote paths, then copy basenames + vlurp 'pkg:github//@#' DEST Pin source in the PURL + vlurp SOURCE SOURCE... DEST Copy multiple remote sources + vlurp SOURCE DEST --glob '**/*.md' Copy files matching a pattern + vlurp SOURCE DEST --iglob '**/skill.md' Match a pattern without case + +Repository preset and filter mode: vlurp / Fetch to .// vlurp / --ref Fetch pinned to commit vlurp / --as -d ./skills Fetch to ./skills/ @@ -121,10 +142,15 @@ ${Object.entries(PRESETS) .join('\n')} Examples: + vlurp mattpocock/skills --preset skills Fetch using a maintained preset vlurp user/repo --ref abc1234 Fetch pinned to commit vlurp user/repo --as myskill -d ./sk Fetch to ./sk/myskill vlurp user/repo --preset claude Use claude preset filters vlurp user/repo --auto Auto-detect structure + vlurp 'pkg:github/mattpocock/skills#skills/in-progress/writing-*' ./skills/ + vlurp 'pkg:github/anthropics/skills@main#skills/pdf' ./skills/ + vlurp 'pkg:github/mattpocock/skills#skills' ./skills/ --preset skills + vlurp 'pkg:github/user/repo#docs' ./docs/ --glob '**/*.md' --glob '!README.md' vlurp batch .vlurpfile Process batch file vlurp batch .vlurpfile --dry-run Preview batch operations vlurp upgrade Upgrade all outdated @@ -142,6 +168,8 @@ Examples: .vlurpfile Format: # Comments start with # + vlurp 'pkg:github/user/repo@abc1234#skills/*' ./skills/ + vlurp pkg:github/user/repo ./docs/ --glob '**/*.md' --iglob '!**/draft-*' vlurp user/repo -d ./vlurp --ref abc1234 vlurp user/repo -d ./vlurp --filter "claude/**" --as myname vlurp user/repo --preset skills @@ -163,6 +191,13 @@ const { ref } = values; const asName = values.as; const dryRun = values['dry-run']; +if (preset && !PRESETS[preset]) { + console.error( + `Error: unknown preset "${preset}"; choose one of: ${Object.keys(PRESETS).join(', ')}` + ); + process.exit(1); +} + // Handle subcommands switch (command) { case 'batch': { @@ -262,11 +297,54 @@ switch (command) { } default: { - // Regular fetch command + if (command.startsWith('pkg:')) { + if (positionals.length < 2) { + console.error('Error: cp-style invocation requires SOURCE and DEST operands'); + console.error("Usage: vlurp 'pkg:github//#' "); + process.exit(1); + } + + const repositoryOptions = [ + rootDir && '-d', + values.filter?.length > 0 && '--filter', + ref && '--ref', + asName && '--as', + auto && '--auto' + ].filter(Boolean); + if (repositoryOptions.length > 0) { + console.error( + `Error: ${repositoryOptions.join(', ')} cannot be used with PURL sources; put remote selections in PURL operands and use --glob or --iglob for transfer patterns` + ); + process.exit(1); + } + + const sources = positionals.slice(0, -1); + const destination = positionals.at(-1); + render( + React.createElement(CopyCommand, { + sources, + destination, + force, + dryRun, + quiet, + globs: values.glob || [], + iglobs: values.iglob || [], + preset + }) + ); + break; + } + + // Repository preset and filter mode const source = command; + if (values.glob?.length > 0 || values.iglob?.length > 0) { + console.error('Error: --glob and --iglob require cp-style PURL SOURCE... DEST operands'); + process.exit(1); + } + // Resolve filters from preset or explicit filters - let filters = values.filter; + let filters = values.filter?.length > 0 ? values.filter : DEFAULT_FILTERS; if (preset && PRESETS[preset]) { filters = PRESETS[preset].filters; } diff --git a/src/commands/batch.js b/src/commands/batch.js index a99dd12..ca813e6 100644 --- a/src/commands/batch.js +++ b/src/commands/batch.js @@ -4,6 +4,7 @@ import process from 'node:process'; import { Box, Text } from 'ink'; import Spinner from 'ink-spinner'; import React, { useEffect, useState } from 'react'; +import { copySources } from '../copy-sources.js'; import { appendLineage, createLineageRecord, hashDirectory } from '../lineage.js'; import { fetchRepository, parseSource } from '../remote.js'; import { parseVlurpfile } from '../vlurpfile.js'; @@ -30,6 +31,15 @@ export function BatchCommand({ vlurpfile, dryRun, force, quiet: _quiet }) { setStatus('dry-run'); const dryResults = parsed.map(entry => { try { + if (entry.mode === 'copy') { + return { + ...entry, + status: 'would-fetch', + targetPath: resolve(entry.destination), + message: `Would copy ${entry.sources.length} source(s) to ${entry.destination}` + }; + } + const parsedSource = parseSource(entry.source, { ref: entry.ref }); const targetPath = entry.targetPath || @@ -60,6 +70,25 @@ export function BatchCommand({ vlurpfile, dryRun, force, quiet: _quiet }) { setCurrentIndex(i); try { + if (entry.mode === 'copy') { + const copyResult = await copySources({ + sources: entry.sources, + destination: entry.destination, + force: force || entry.force, + globs: entry.globs, + iglobs: entry.iglobs, + preset: entry.preset + }); + batchResults.push({ + ...entry, + status: copyResult.status === 'complete' ? 'success' : copyResult.status, + targetPath: resolve(entry.destination), + message: `${entry.sources.length} source(s) -> ${entry.destination}` + }); + setResults([...batchResults]); + continue; + } + const parsedSource = parseSource(entry.source, { ref: entry.ref }); const targetPath = entry.targetPath || diff --git a/src/commands/copy.js b/src/commands/copy.js new file mode 100644 index 0000000..82d2a94 --- /dev/null +++ b/src/commands/copy.js @@ -0,0 +1,86 @@ +import process from 'node:process'; +import { Box, Text } from 'ink'; +import Spinner from 'ink-spinner'; +import React, { useEffect, useState } from 'react'; +import { copySources } from '../copy-sources.js'; + +export function CopyCommand({ sources, destination, force, dryRun, quiet, globs, iglobs, preset }) { + const [status, setStatus] = useState('copying'); + const [error, setError] = useState(null); + const [result, setResult] = useState(null); + + useEffect(() => { + async function performCopy() { + try { + const copyResult = await copySources({ + sources, + destination, + force, + dryRun, + globs, + iglobs, + preset + }); + setResult(copyResult); + setStatus(copyResult.status); + } catch (copyError) { + process.exitCode = 1; + setError(copyError.message); + setStatus('error'); + } + } + + performCopy(); + }, [sources, destination, force, dryRun, globs, iglobs, preset]); + + if (status === 'error') { + return React.createElement(Text, { color: 'red' }, `Error: ${error}`); + } + + if (status === 'cancelled') { + return React.createElement(Text, { color: 'yellow' }, 'vlurping cancelled.'); + } + + if (status === 'dry-run') { + return React.createElement( + Box, + { flexDirection: 'column' }, + React.createElement(Text, { color: 'yellow', bold: true }, 'Dry run - would copy:'), + ...result.plan.entries.map(entry => + React.createElement( + Text, + { key: `${entry.source.raw}:${entry.relativePath}`, color: 'gray' }, + ` ${entry.source.raw} -> ${entry.targetPath}` + ) + ) + ); + } + + if (status === 'complete') { + return React.createElement( + Box, + { flexDirection: 'column' }, + React.createElement( + Text, + { color: 'green' }, + `vlurped ${result.fileCount} file(s) to ${result.plan.destinationPath}` + ), + ...(!quiet + ? result.plan.entries.map(entry => + React.createElement( + Text, + { key: entry.targetPath, color: 'gray' }, + ` ${entry.targetPath}` + ) + ) + : []) + ); + } + + return React.createElement( + Box, + null, + React.createElement(Text, null, React.createElement(Spinner, { type: 'dots' })), + React.createElement(Text, null, ' Resolving remote sources...') + ); +} diff --git a/src/commands/outdated.js b/src/commands/outdated.js index d7fd663..1a60ce0 100644 --- a/src/commands/outdated.js +++ b/src/commands/outdated.js @@ -1,9 +1,9 @@ import { readFile } from 'node:fs/promises'; -import { join, resolve } from 'node:path'; +import { resolve } from 'node:path'; import { Box, Text } from 'ink'; import Spinner from 'ink-spinner'; import React, { useEffect, useState } from 'react'; -import { readLineage } from '../lineage.js'; +import { expandManifestSources } from '../manifest-sources.js'; import { Fetcher } from '../remote.js'; import { parseVlurpfile } from '../vlurpfile.js'; @@ -32,40 +32,14 @@ export function OutdatedCommand({ vlurpfilePath }) { return; } - // Try to read lineage for additional context - const lineageRecords = []; - for (const entry of entries) { - if (entry.rootDir) { - const lineagePath = join(resolve(entry.rootDir), '.vlurp.jsonl'); - try { - const records = await readLineage(lineagePath); - lineageRecords.push(...records); - } catch { - // No lineage file yet - } - } - } - const fetcher = new Fetcher(); const outdatedResults = []; - for (const entry of entries) { - const parts = entry.source.split('/'); - if (parts.length < 2) { - outdatedResults.push({ - source: entry.source, - status: 'error', - message: 'Invalid source format' - }); - continue; - } - - const [user, repo] = parts; - - const upstreamSha = await fetcher.resolveHead(user, repo); + for (const source of expandManifestSources(entries)) { + const upstreamSha = await fetcher.resolveHead(source.user, source.repo); if (!upstreamSha) { outdatedResults.push({ - source: entry.source, + source: source.raw, status: 'error', message: 'Could not resolve upstream HEAD' }); @@ -74,29 +48,29 @@ export function OutdatedCommand({ vlurpfilePath }) { const shortUpstream = upstreamSha.slice(0, 7); - if (!entry.ref) { + if (!source.ref) { outdatedResults.push({ - source: entry.source, + source: source.raw, status: 'unpinned', upstream: shortUpstream, message: 'Not pinned — always fetches latest' }); } else if ( - upstreamSha.startsWith(entry.ref) || - entry.ref.startsWith(upstreamSha.slice(0, entry.ref.length)) + upstreamSha.startsWith(source.ref) || + source.ref.startsWith(upstreamSha.slice(0, source.ref.length)) ) { // Compare short SHAs sensibly outdatedResults.push({ - source: entry.source, + source: source.raw, status: 'current', - pinned: entry.ref, + pinned: source.ref, upstream: shortUpstream }); } else { outdatedResults.push({ - source: entry.source, + source: source.raw, status: 'outdated', - pinned: entry.ref, + pinned: source.ref, upstream: shortUpstream }); } diff --git a/src/commands/pin.js b/src/commands/pin.js index ed41ef2..a53dcef 100644 --- a/src/commands/pin.js +++ b/src/commands/pin.js @@ -3,8 +3,9 @@ import { resolve } from 'node:path'; import { Box, Text } from 'ink'; import Spinner from 'ink-spinner'; import React, { useEffect, useState } from 'react'; +import { expandManifestSources, manifestSourceMatches } from '../manifest-sources.js'; import { Fetcher } from '../remote.js'; -import { parseVlurpfile } from '../vlurpfile.js'; +import { parseVlurpfile, updateRef } from '../vlurpfile.js'; export function PinCommand({ source, vlurpfilePath }) { const [status, setStatus] = useState('resolving'); @@ -28,9 +29,10 @@ export function PinCommand({ source, vlurpfilePath }) { const pinResults = []; // Filter to specific source if provided + const manifestSources = expandManifestSources(entries); const toPin = source - ? entries.filter(e => e.source === source || e.source.includes(source)) - : entries.filter(e => !e.ref); + ? manifestSources.filter(item => manifestSourceMatches(item, source) && !item.ref) + : manifestSources.filter(item => !item.ref); if (toPin.length === 0) { if (source) { @@ -45,23 +47,11 @@ export function PinCommand({ source, vlurpfilePath }) { let updatedContent = content; - for (const entry of toPin) { - const parts = entry.source.split('/'); - if (parts.length < 2) { - pinResults.push({ - source: entry.source, - status: 'error', - message: 'Invalid source format' - }); - continue; - } - - const [user, repo] = parts; - - const sha = await fetcher.resolveHead(user, repo); + for (const item of toPin) { + const sha = await fetcher.resolveHead(item.user, item.repo); if (!sha) { pinResults.push({ - source: entry.source, + source: item.raw, status: 'error', message: 'Could not resolve HEAD' }); @@ -70,23 +60,9 @@ export function PinCommand({ source, vlurpfilePath }) { const shortSha = sha.slice(0, 7); - // Update the vlurpfile content — find the line with this source and add --ref - const sourcePattern = new RegExp( - `(vlurp\\s+${escapeRegex(entry.source)}(?:\\s+[^\\n]*?)?)(?:\\s*(?:#.*)?)$`, - 'gm' - ); - - updatedContent = updatedContent.replace(sourcePattern, (match, command) => { - // Don't add ref if already pinned - if (command.includes('--ref ')) { - return match; - } - - const comment = match.slice(command.length); - return `${command} --ref ${shortSha}${comment}`; - }); + updatedContent = updateRef(updatedContent, item.raw, shortSha); - pinResults.push({ source: entry.source, status: 'pinned', sha: shortSha }); + pinResults.push({ source: item.raw, status: 'pinned', sha: shortSha }); setResults([...pinResults]); } @@ -163,7 +139,3 @@ async function findVlurpfile(explicitPath) { return null; } - -function escapeRegex(string) { - return string.replaceAll(/[.*+?^${}()|[\]\\]/g, String.raw`\$&`); -} diff --git a/src/commands/upgrade.js b/src/commands/upgrade.js index c820a3b..c085fca 100644 --- a/src/commands/upgrade.js +++ b/src/commands/upgrade.js @@ -1,13 +1,17 @@ +import { statSync } from 'node:fs'; import { readFile, rename, writeFile } from 'node:fs/promises'; -import { join, resolve } from 'node:path'; +import { dirname, join, resolve } from 'node:path'; import process from 'node:process'; import { Box, Text } from 'ink'; import Spinner from 'ink-spinner'; import React, { useEffect, useState } from 'react'; import { buildCatalog } from '../catalog.js'; import { diffCatalogs, formatCatalogDiff } from '../catalog-diff.js'; +import { copySources } from '../copy-sources.js'; import { appendLineage, createLineageRecord, hashDirectory } from '../lineage.js'; +import { expandManifestSources, manifestSourceMatches } from '../manifest-sources.js'; import { Fetcher, fetchRepository, parseSource } from '../remote.js'; +import { replaceSourceRef } from '../source-operand.js'; import { parseVlurpfile, updateRefs } from '../vlurpfile.js'; export function UpgradeCommand({ vlurpfilePath, source, dryRun }) { @@ -104,23 +108,25 @@ async function runUpgrade(options) { return; } - // Filter to specific source if given - const targetEntries = source ? entries.filter(e => e.source === source) : entries; + const manifestSources = expandManifestSources(entries); + const targetSources = source + ? manifestSources.filter(item => manifestSourceMatches(item, source)) + : manifestSources; - if (source && targetEntries.length === 0) { + if (source && targetSources.length === 0) { setError(`Source "${source}" not found in vlurpfile`); setStatus('error'); return; } - // Deduplicate by source + // Group selectors from one repository so HEAD is resolved and fetched once. const sourceMap = new Map(); - for (const entry of targetEntries) { - if (!sourceMap.has(entry.source)) { - sourceMap.set(entry.source, []); + for (const item of targetSources) { + if (!sourceMap.has(item.repositoryKey)) { + sourceMap.set(item.repositoryKey, []); } - sourceMap.get(entry.source).push(entry); + sourceMap.get(item.repositoryKey).push(item); } // Step 2: Check upstream @@ -141,7 +147,7 @@ async function runUpgrade(options) { const refUpdates = {}; for (const r of upgradeResults) { if (r.status === 'upgraded') { - refUpdates[r.source] = r.newRef; + Object.assign(refUpdates, r.refUpdates); } } @@ -165,20 +171,14 @@ async function runUpgrade(options) { async function checkAndUpgradeSources(options) { const { sourceMap, fetcher, dryRun, entries, setCurrentSource, setResults, setStatus } = options; const upgradeResults = []; + const resolvedRefs = new Map(); // Collect rootDirs for catalog snapshot const rootDirs = collectRootDirs(entries); - for (const [src, srcEntries] of sourceMap) { + for (const [src, sourceItems] of sourceMap) { setCurrentSource(src); - const parts = src.split('/'); - if (parts.length < 2) { - upgradeResults.push({ source: src, status: 'error', message: 'Invalid source format' }); - setResults([...upgradeResults]); - continue; - } - - const [user, repo] = parts; + const { user, repo } = sourceItems[0]; const upstreamSha = await fetcher.resolveHead(user, repo); if (!upstreamSha) { @@ -190,12 +190,17 @@ async function checkAndUpgradeSources(options) { setResults([...upgradeResults]); continue; } + resolvedRefs.set(src, upstreamSha); - const pinnedRef = srcEntries[0].ref; + const pinnedRefs = sourceItems.map(item => item.ref).filter(Boolean); + const pinnedRef = pinnedRefs.length === 1 ? pinnedRefs[0] : pinnedRefs.join(','); const isCurrent = - pinnedRef && - (upstreamSha.startsWith(pinnedRef) || - pinnedRef.startsWith(upstreamSha.slice(0, pinnedRef.length))); + pinnedRefs.length === sourceItems.length && + pinnedRefs.every( + candidate => + upstreamSha.startsWith(candidate) || + candidate.startsWith(upstreamSha.slice(0, candidate.length)) + ); if (isCurrent) { upgradeResults.push({ @@ -214,7 +219,7 @@ async function checkAndUpgradeSources(options) { status: 'outdated', ref: pinnedRef || null, upstream: upstreamSha.slice(0, 7), - entries: srcEntries.length + entries: sourceItems.length }); setResults([...upgradeResults]); continue; @@ -226,7 +231,18 @@ async function checkAndUpgradeSources(options) { // Fetch each entry for this source with the new ref setStatus('upgrading'); const shortSha = upstreamSha.slice(0, 7); - const fetchedAll = await fetchSourceEntries(srcEntries, upstreamSha, setCurrentSource); + const sourceEntries = [...new Set(sourceItems.map(item => item.entry))]; + const fetchedAll = await fetchSourceEntries( + sourceEntries, + upstreamSha, + resolvedRefs, + setCurrentSource + ); + if (!fetchedAll) { + // Do not let a later multi-repository command record this failed + // repository at a ref that its manifest will not receive. + resolvedRefs.delete(src); + } // Step 4: Snapshot post-upgrade catalog and diff const postCatalog = await snapshotCatalogs(rootDirs, setStatus, setCurrentSource); @@ -241,7 +257,8 @@ async function checkAndUpgradeSources(options) { ref: pinnedRef || null, upstream: shortSha, newRef: shortSha, - entries: srcEntries.length, + entries: sourceItems.length, + refUpdates: Object.fromEntries(sourceItems.map(item => [item.raw, shortSha])), message: fetchedAll ? null : 'One or more entries failed to fetch', catalogDiff }); @@ -254,12 +271,27 @@ async function checkAndUpgradeSources(options) { function collectRootDirs(entries) { const dirs = new Set(); for (const entry of entries) { - dirs.add(entry.rootDir ? resolve(entry.rootDir) : process.cwd()); + dirs.add( + entry.mode === 'copy' + ? copyLineageRoot(entry.destination) + : entry.rootDir + ? resolve(entry.rootDir) + : process.cwd() + ); } return [...dirs]; } +function copyLineageRoot(destination) { + const path = resolve(destination); + try { + return statSync(path).isDirectory() ? path : dirname(path); + } catch { + return dirname(path); + } +} + async function snapshotCatalogs(rootDirs, setStatus, setCurrentSource) { setStatus('cataloging'); setCurrentSource('building catalog...'); @@ -304,10 +336,33 @@ async function saveCatalogs(rootDirs, preCatalog, postCatalog) { } } -async function fetchSourceEntries(srcEntries, upstreamSha, setCurrentSource) { +async function fetchSourceEntries(srcEntries, upstreamSha, resolvedRefs, setCurrentSource) { let fetchedAll = true; for (const entry of srcEntries) { + if (entry.mode === 'copy') { + setCurrentSource(entry.sources.join(', ')); + try { + const sources = entry.sources.map(source => { + const item = expandManifestSources([{ ...entry, sources: [source] }])[0]; + const resolvedRef = resolvedRefs.get(item.repositoryKey); + return resolvedRef ? replaceSourceRef(source, resolvedRef) : source; + }); + await copySources({ + sources, + destination: entry.destination, + force: true, + globs: entry.globs, + iglobs: entry.iglobs, + preset: entry.preset + }); + } catch { + fetchedAll = false; + } + + continue; + } + setCurrentSource(`${entry.source}${entry.as ? ` (${entry.as})` : ''}`); try { const parsed = parseSource(entry.source, { ref: upstreamSha }); diff --git a/src/copy-plan.js b/src/copy-plan.js new file mode 100644 index 0000000..8ba378a --- /dev/null +++ b/src/copy-plan.js @@ -0,0 +1,355 @@ +import { cp, lstat, mkdir, readdir } from 'node:fs/promises'; +import { basename, dirname, isAbsolute, relative, resolve, sep } from 'node:path'; +import { glob } from 'glob'; +import { compileTransferRules } from './transfer-rules.js'; + +export async function selectRepositoryEntries(repositoryRoot, source) { + const root = resolve(repositoryRoot); + if (!source.subpath) { + return [await selectionFor(root, source.repo || basename(root), '', source)]; + } + + assertSafeRelativePath(source.subpath); + if (!source.pattern) { + const sourcePath = resolve(root, ...source.subpath.split('/')); + assertContained(root, sourcePath); + return [await selectionFor(sourcePath, basename(source.subpath), source.subpath, source)]; + } + + const matches = await glob(source.subpath, { + cwd: root, + dot: true, + nodir: false, + posix: true + }); + const sorted = [...new Set(matches.map(normalizeRelative))].sort(); + const collapsed = []; + for (const match of sorted) { + if (collapsed.some(parent => match.startsWith(`${parent}/`))) { + continue; + } + + collapsed.push(match); + } + + if (collapsed.length === 0) { + throw new Error(`Source pattern matched no files: ${source.raw || source.subpath}`); + } + + return Promise.all( + collapsed.map(match => { + const sourcePath = resolve(root, ...match.split('/')); + assertContained(root, sourcePath); + return selectionFor(sourcePath, basename(match), match, source); + }) + ); +} + +export async function createCopyPlan(selections, destination, transferOptions = {}) { + if (!Array.isArray(selections) || selections.length === 0) { + throw new Error('At least one source selection is required'); + } + + const destinationPath = resolve(destination); + const destinationStat = await optionalLstat(destinationPath); + const destinationIsDirectory = destinationStat?.isDirectory() || false; + if (selections.length > 1 && !destinationIsDirectory) { + throw new Error('When copying multiple sources, DEST must be an existing directory'); + } + + const entries = []; + const targets = new Set(); + const transferRules = compileTransferRules(transferOptions); + for (const selection of selections) { + await validateSelectionTree(selection.sourcePath); + const transferFiles = transferRules.active + ? await selectTransferFiles(selection, transferRules) + : null; + if (transferFiles && transferFiles.length === 0) { + continue; + } + if (destinationStat && !destinationIsDirectory && selection.kind === 'directory') { + throw new Error( + `Cannot overwrite non-directory DEST with directory ${selection.relativePath}` + ); + } + + const targetPath = destinationIsDirectory + ? resolve(destinationPath, selection.basename) + : destinationPath; + assertContained( + destinationIsDirectory ? destinationPath : dirname(destinationPath), + targetPath, + { + allowRoot: !destinationIsDirectory + } + ); + if (targets.has(targetPath)) { + throw new Error(`Source basename collision at destination: ${selection.basename}`); + } + + const targetStat = await optionalLstat(targetPath); + if (targetStat?.isSymbolicLink()) { + throw new Error(`Refusing to overwrite symbolic link at DEST: ${targetPath}`); + } + + if (targetStat && !targetStat.isFile() && !targetStat.isDirectory()) { + throw new Error(`Refusing to overwrite special filesystem entry at DEST: ${targetPath}`); + } + + if ( + targetStat && + ((selection.kind === 'directory' && !targetStat.isDirectory()) || + (selection.kind === 'file' && !targetStat.isFile())) + ) { + throw new Error(`Source and destination types differ at ${targetPath}`); + } + + if (transferFiles && selection.kind === 'directory') { + await validateTransferTargets(targetPath, transferFiles); + } + + targets.add(targetPath); + entries.push({ + ...selection, + targetPath, + exists: Boolean(targetStat), + transferFiles + }); + } + + if (entries.length === 0) { + throw new Error('Transfer rules matched no files beneath the selected sources'); + } + + const plan = { + destinationPath, + destinationExisted: Boolean(destinationStat), + destinationIsDirectory, + lineageRoot: + destinationIsDirectory || (!destinationStat && selections[0].kind === 'directory') + ? destinationPath + : dirname(destinationPath), + entries, + transferRules: { + globs: transferRules.globs, + iglobs: transferRules.iglobs, + preset: transferRules.preset + } + }; + await validateReservedTargets(plan); + return plan; +} + +export async function executeCopyPlan(plan) { + for (const entry of plan.entries) { + if (entry.transferFiles) { + if (entry.kind === 'directory') { + await mkdir(entry.targetPath, { recursive: true }); + } + + for (const file of entry.transferFiles) { + const sourcePath = file ? resolve(entry.sourcePath, ...file.split('/')) : entry.sourcePath; + const targetPath = file ? resolve(entry.targetPath, ...file.split('/')) : entry.targetPath; + await mkdir(dirname(targetPath), { recursive: true }); + await cp(sourcePath, targetPath, { force: true, errorOnExist: false }); + } + + continue; + } + + await mkdir(dirname(entry.targetPath), { recursive: true }); + await cp(entry.sourcePath, entry.targetPath, { + recursive: entry.kind === 'directory', + force: true, + errorOnExist: false + }); + } +} + +export async function listCopiedFiles(plan) { + const files = []; + for (const entry of plan.entries) { + if (entry.transferFiles) { + for (const file of entry.transferFiles) { + const targetPath = file ? resolve(entry.targetPath, ...file.split('/')) : entry.targetPath; + files.push(relative(plan.lineageRoot, targetPath)); + } + + continue; + } + + if (entry.kind === 'file') { + files.push(relative(plan.lineageRoot, entry.targetPath)); + continue; + } + + await walk(entry.targetPath, path => files.push(relative(plan.lineageRoot, path))); + } + + return [...new Set(files)].sort(); +} + +async function selectTransferFiles(selection, rules) { + if (selection.kind === 'file') { + return rules.matches(selection.basename) ? [''] : []; + } + + const files = []; + await walk(selection.sourcePath, path => { + const relativePath = normalizeRelative(relative(selection.sourcePath, path)); + if (rules.matches(relativePath)) { + files.push(relativePath); + } + }); + return files.sort(); +} + +async function selectionFor(sourcePath, selectedBasename, relativePath, source) { + const stat = await optionalLstat(sourcePath); + if (!stat) { + throw new Error(`Source path does not exist: ${relativePath || '.'}`); + } + + if (stat.isSymbolicLink()) { + throw new Error(`Refusing to copy symbolic link: ${relativePath}`); + } + + if (!stat.isFile() && !stat.isDirectory()) { + throw new Error(`Unsupported source entry type: ${relativePath}`); + } + + return { + source, + sourcePath, + relativePath, + basename: selectedBasename, + kind: stat.isDirectory() ? 'directory' : 'file' + }; +} + +async function walk(directory, onFile) { + const entries = await readdir(directory, { withFileTypes: true }); + for (const entry of entries) { + const path = resolve(directory, entry.name); + if (entry.isSymbolicLink()) { + throw new Error(`Refusing to record symbolic link: ${path}`); + } + + if (entry.isDirectory()) { + await walk(path, onFile); + } else if (entry.isFile()) { + onFile(path); + } + } +} + +async function validateSelectionTree(path) { + const stat = await lstat(path); + if (stat.isSymbolicLink()) { + throw new Error(`Refusing to copy symbolic link: ${path}`); + } + + if (stat.isFile()) { + return; + } + + if (!stat.isDirectory()) { + throw new Error(`Refusing to copy special filesystem entry: ${path}`); + } + + const entries = await readdir(path); + for (const entry of entries) { + await validateSelectionTree(resolve(path, entry)); + } +} + +async function validateReservedTargets(plan) { + for (const reservedName of ['.vlurp.jsonl', '.vlurp.sigstore']) { + const reservedPath = resolve(plan.lineageRoot, reservedName); + for (const entry of plan.entries) { + if (entry.kind === 'file' && entry.targetPath === reservedPath) { + throw new Error(`Source would overwrite reserved vlurp metadata: ${reservedName}`); + } + + if (entry.kind !== 'directory') { + continue; + } + + const mappedRelative = relative(entry.targetPath, reservedPath); + if ( + mappedRelative === '..' || + mappedRelative.startsWith(`..${sep}`) || + isAbsolute(mappedRelative) + ) { + continue; + } + + if (entry.transferFiles && !entry.transferFiles.includes(normalizeRelative(mappedRelative))) { + continue; + } + + if (await optionalLstat(resolve(entry.sourcePath, mappedRelative))) { + throw new Error(`Source would overwrite reserved vlurp metadata: ${reservedName}`); + } + } + } +} + +async function validateTransferTargets(targetRoot, transferFiles) { + for (const file of transferFiles) { + const segments = file.split('/'); + let path = targetRoot; + for (const [index, segment] of segments.entries()) { + path = resolve(path, segment); + assertContained(targetRoot, path); + const stat = await optionalLstat(path); + if (!stat) { + continue; + } + + if (stat.isSymbolicLink()) { + throw new Error(`Refusing to copy through symbolic link at DEST: ${path}`); + } + + const isFile = index === segments.length - 1; + if ((isFile && !stat.isFile()) || (!isFile && !stat.isDirectory())) { + throw new Error(`Source and destination types differ at ${path}`); + } + } + } +} + +function assertSafeRelativePath(path) { + if (isAbsolute(path) || path.startsWith('\\')) { + throw new Error(`Unsafe repository subpath: ${path}`); + } + + const segments = path.split('/'); + if (segments.some(segment => segment === '..' || segment === '.' || segment === '')) { + throw new Error(`Unsafe repository subpath: ${path}`); + } +} + +function assertContained(root, path, { allowRoot = true } = {}) { + const rel = relative(resolve(root), resolve(path)); + if ((!allowRoot && rel === '') || rel === '..' || rel.startsWith(`..${sep}`) || isAbsolute(rel)) { + throw new Error(`Unsafe path escapes destination or repository root: ${path}`); + } +} + +function normalizeRelative(path) { + return path.replaceAll('\\', '/').replace(/\/$/, ''); +} + +async function optionalLstat(path) { + try { + return await lstat(path); + } catch (error) { + if (error.code === 'ENOENT') { + return null; + } + + throw error; + } +} diff --git a/src/copy-sources.js b/src/copy-sources.js new file mode 100644 index 0000000..b7779a0 --- /dev/null +++ b/src/copy-sources.js @@ -0,0 +1,97 @@ +import { join, relative } from 'node:path'; +import process from 'node:process'; +import { createInterface } from 'node:readline'; +import { + createCopyPlan, + executeCopyPlan, + listCopiedFiles, + selectRepositoryEntries +} from './copy-plan.js'; +import { appendLineage, createCopyLineageRecord, hashFile } from './lineage.js'; +import { Fetcher } from './remote.js'; +import { parseSourceOperand, sourceRepositoryKey } from './source-operand.js'; + +export async function copySources({ + sources, + destination, + globs = [], + iglobs = [], + preset = null, + force = false, + dryRun = false, + confirm = confirmOverwrite, + fetcher = new Fetcher() +}) { + const parsedSources = sources.map(parseSourceOperand); + const materialized = new Map(); + + try { + const selections = []; + for (const source of parsedSources) { + const materializationKey = `${sourceRepositoryKey(source)}@${source.ref || 'HEAD'}`; + let repository = materialized.get(materializationKey); + if (!repository) { + repository = await fetcher.materialize(source.tarballUrl); + materialized.set(materializationKey, repository); + } + + selections.push(...(await selectRepositoryEntries(repository.path, source))); + } + + const plan = await createCopyPlan(selections, destination, { globs, iglobs, preset }); + const existing = plan.entries.filter(entry => entry.exists); + if (!dryRun && existing.length > 0 && !force) { + const accepted = await confirm(existing.map(entry => entry.targetPath)); + if (!accepted) { + return { status: 'cancelled', plan, fileCount: 0, records: [] }; + } + } + + if (dryRun) { + return { status: 'dry-run', plan, fileCount: 0, records: [] }; + } + + await executeCopyPlan(plan); + const records = []; + for (const source of parsedSources) { + const sourceEntries = plan.entries.filter(entry => entry.source === source); + const sourcePlan = { ...plan, entries: sourceEntries }; + const copiedFiles = await listCopiedFiles(sourcePlan); + const files = {}; + for (const file of copiedFiles) { + files[file] = await hashFile(join(plan.lineageRoot, file)); + } + + const record = createCopyLineageRecord({ + source: source.raw, + sourceKey: `${sourceRepositoryKey(source)}#${source.subpath}`, + ref: source.ref, + destination: relative(plan.lineageRoot, plan.destinationPath) || '.', + globs, + iglobs, + preset, + files + }); + await appendLineage(join(plan.lineageRoot, '.vlurp.jsonl'), record); + records.push(record); + } + + return { + status: 'complete', + plan, + fileCount: new Set(records.flatMap(record => Object.keys(record.files))).size, + records + }; + } finally { + await Promise.all([...materialized.values()].map(repository => repository.cleanup())); + } +} + +async function confirmOverwrite(paths) { + console.log(`\nWarning: ${paths.length} destination path(s) already exist.`); + console.log('Continuing will overwrite matching files.'); + const readline = createInterface({ input: process.stdin, output: process.stdout }); + const answer = await new Promise(resolve => readline.question('Continue? (y/N): ', resolve)); + readline.close(); + return answer.toLowerCase() === 'y'; +} diff --git a/src/index.js b/src/index.js index 01421ee..2e431e5 100644 --- a/src/index.js +++ b/src/index.js @@ -3,9 +3,17 @@ export { diffCatalogs, formatCatalogDiff } from './catalog-diff.js'; +export { + createCopyPlan, + executeCopyPlan, + listCopiedFiles, + selectRepositoryEntries +} from './copy-plan.js'; +export { copySources } from './copy-sources.js'; export { detectStructure } from './detector.js'; export { appendLineage, + createCopyLineageRecord, createLineageRecord, hashDirectory, hashFile, @@ -26,4 +34,10 @@ export { scanFileContent, summarizeScan } from './scanner.js'; +export { + parseSourceOperand, + replaceSourceRef, + sourceRepositoryKey +} from './source-operand.js'; +export { compileTransferRules } from './transfer-rules.js'; export { parseVlurpfile, updateRef, updateRefs } from './vlurpfile.js'; diff --git a/src/lineage.js b/src/lineage.js index 9862203..7e845d6 100644 --- a/src/lineage.js +++ b/src/lineage.js @@ -53,6 +53,37 @@ export function createLineageRecord({ source, ref, refType, filters, preset, asN }; } +/** + * Create a version 2 lineage record for cp-style source operands. File keys + * are relative to the directory containing .vlurp.jsonl, so verification no + * longer has to infer layout from a repository name or legacy --as flag. + */ +export function createCopyLineageRecord({ + source, + sourceKey, + ref, + destination, + globs = [], + iglobs = [], + preset = null, + files +}) { + return { + schema: 2, + source, + lineage_key: sourceKey, + ref: ref || null, + ref_type: ref ? 'git' : null, + fetched_at: new Date().toISOString(), + destination, + globs, + iglobs, + preset, + files_root_relative: true, + files + }; +} + /** * Append a lineage record to a .vlurp.jsonl file. */ @@ -64,15 +95,32 @@ export async function appendLineage(jsonlPath, record) { // File doesn't exist yet } - // Replace existing record for the same source+as combo, or append + // Replace the same source/destination record, then transfer ownership of + // any overwritten root-relative paths to the newest record. const lines = existing.split('\n').filter(l => l.trim()); const key = recordKey(record); - const filtered = lines.filter(l => { + const claimedFiles = new Set(record.files_root_relative ? Object.keys(record.files || {}) : []); + const filtered = lines.flatMap(line => { try { - const parsed = JSON.parse(l); - return recordKey(parsed) !== key; + const parsed = JSON.parse(line); + if (recordKey(parsed) === key) { + return []; + } + + if (record.files_root_relative && parsed.files_root_relative) { + const remainingFiles = Object.fromEntries( + Object.entries(parsed.files || {}).filter(([file]) => !claimedFiles.has(file)) + ); + if (Object.keys(remainingFiles).length === 0) { + return []; + } + + return [JSON.stringify({ ...parsed, files: remainingFiles })]; + } + + return [line]; } catch { - return true; + return [line]; } }); @@ -118,7 +166,9 @@ export async function verifyFiles(basePath, records) { // If --as was used, files are under {as}/ // Otherwise, files are under {user}/{repo}/ derived from source let prefix = ''; - if (record.as) { + if (record.files_root_relative) { + prefix = ''; + } else if (record.as) { prefix = record.as; } else { // Source is "github:user/repo" — extract "user/repo" @@ -192,8 +242,8 @@ export async function verifyFiles(basePath, records) { } /** - * Unique key for a lineage record (source + as name). + * Unique key for a lineage record and its destination layout. */ function recordKey(record) { - return `${record.source}::${record.as || ''}`; + return `${record.lineage_key || record.source}::${record.destination || record.as || ''}`; } diff --git a/src/manifest-sources.js b/src/manifest-sources.js new file mode 100644 index 0000000..9db9594 --- /dev/null +++ b/src/manifest-sources.js @@ -0,0 +1,45 @@ +import { parseSource } from './remote.js'; +import { parseSourceOperand, sourceRepositoryKey } from './source-operand.js'; + +/** Flatten vlurpfile commands into independently pinnable repository sources. */ +export function expandManifestSources(entries) { + const sources = []; + for (const entry of entries) { + if (entry.mode === 'copy') { + for (const raw of entry.sources) { + const parsed = parseSourceOperand(raw); + sources.push({ + entry, + raw, + user: parsed.owner, + repo: parsed.repo, + ref: parsed.ref, + repositoryKey: sourceRepositoryKey(parsed), + purl: parsed + }); + } + } else { + const parsed = parseSource(entry.source, { ref: entry.ref }); + sources.push({ + entry, + raw: entry.source, + user: parsed.user, + repo: parsed.repo, + ref: entry.ref, + repositoryKey: `github:${parsed.user}/${parsed.repo}`, + purl: null + }); + } + } + + return sources; +} + +export function manifestSourceMatches(source, query) { + return ( + source.raw === query || + source.repositoryKey === query || + `${source.user}/${source.repo}` === query || + source.raw.includes(query) + ); +} diff --git a/src/remote.js b/src/remote.js index d54130d..ce1c54f 100644 --- a/src/remote.js +++ b/src/remote.js @@ -2,7 +2,7 @@ import { randomBytes } from 'node:crypto'; import { constants, createWriteStream } from 'node:fs'; import { access, cp, mkdir, readdir, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; -import { dirname, join } from 'node:path'; +import { dirname, isAbsolute, join } from 'node:path'; import process from 'node:process'; import { createInterface } from 'node:readline'; import { pipeline } from 'node:stream/promises'; @@ -165,11 +165,7 @@ export class Fetcher { await mkdir(temporaryExtractDir, { recursive: true }); // Extract everything first - await extract({ - file: tarballPath, - cwd: temporaryExtractDir, - strip: 1 // Strip the top-level directory from the tarball - }); + await this.#extractArchive(tarballPath, temporaryExtractDir); // If no filters provided, copy everything if (!filters || filters.length === 0) { @@ -212,6 +208,51 @@ export class Fetcher { await rm(temporaryExtractDir, { recursive: true, force: true }); } + async #extractArchive(tarballPath, temporaryExtractDir) { + await extract({ + file: tarballPath, + cwd: temporaryExtractDir, + strip: 1, + preservePaths: false, + filter: (path, entry) => { + const normalized = path.replaceAll('\\', '/'); + const segments = normalized.split('/'); + if (isAbsolute(path) || normalized.startsWith('/') || segments.includes('..')) { + return false; + } + + // Remote links and device nodes can escape or confer filesystem + // capabilities. vlurp materializes only plain files and directories. + return entry.type === 'File' || entry.type === 'OldFile' || entry.type === 'Directory'; + } + }); + } + + async materialize(tarballUrl) { + const temporaryExtractDir = join(tmpdir(), `vlurp-extract-${randomBytes(8).toString('hex')}`); + let temporaryTarball; + await mkdir(temporaryExtractDir, { recursive: true }); + + try { + temporaryTarball = await this.#downloadTarball(tarballUrl); + await this.#extractArchive(temporaryTarball, temporaryExtractDir); + return { + path: temporaryExtractDir, + cleanup: async () => { + await rm(temporaryExtractDir, { recursive: true, force: true }); + } + }; + } catch (error) { + await rm(temporaryExtractDir, { recursive: true, force: true }); + throw error; + } finally { + if (temporaryTarball) { + const { unlink } = await import('node:fs/promises'); + await unlink(temporaryTarball, { force: true }); + } + } + } + async countFiles(dir) { try { const files = await readdir(dir, { recursive: true }); diff --git a/src/source-operand.js b/src/source-operand.js new file mode 100644 index 0000000..d93baf5 --- /dev/null +++ b/src/source-operand.js @@ -0,0 +1,148 @@ +import { Minimatch } from 'minimatch'; + +export function parseSourceOperand(input) { + if (typeof input !== 'string' || !input.startsWith('pkg:')) { + throw new Error('Remote sources must be Package URLs (PURLs) beginning with "pkg:"'); + } + + const match = /^pkg:([^/]+)\/([^?#]+)(?:\?([^#]*))?(?:#(.*))?$/.exec(input); + if (!match) { + throw new Error(`Malformed PURL source: ${input}`); + } + + const [, rawType, rawPath, rawQualifiers = '', rawSubpath = ''] = match; + const type = decodePart(rawType, 'type').toLowerCase(); + if (type !== 'github') { + throw new Error(`Unsupported PURL type "${type}"; vlurp currently supports pkg:github`); + } + + const at = rawPath.lastIndexOf('@'); + const rawRepository = at === -1 ? rawPath : rawPath.slice(0, at); + const rawRef = at === -1 ? '' : rawPath.slice(at + 1); + if (at !== -1 && !rawRef) { + throw new Error('A PURL @ref must not be empty'); + } + const repositorySegments = rawRepository.split('/'); + if (repositorySegments.length !== 2 || repositorySegments.some(segment => !segment)) { + throw new Error('A pkg:github source must contain exactly one owner and repository'); + } + + if (hasGlobMagic(rawRepository) || hasGlobMagic(rawRef)) { + throw new Error('Glob syntax is only allowed in a PURL #subpath'); + } + + const owner = decodePart(repositorySegments[0], 'owner'); + const repo = decodePart(repositorySegments[1], 'repository'); + const ref = rawRef ? decodePart(rawRef, 'ref') : null; + if (hasGlobMagic(owner) || hasGlobMagic(repo) || hasGlobMagic(ref || '')) { + throw new Error('Glob syntax is only allowed in a PURL #subpath'); + } + const subpath = decodeSubpath(rawSubpath); + validateSubpath(subpath, rawSubpath); + + return { + raw: input, + type, + owner, + repo, + ref, + qualifiers: parseQualifiers(rawQualifiers), + subpath, + pattern: hasGlobMagic(rawSubpath), + tarballUrl: + `https://api.github.com/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/tarball/${ref ? encodeURIComponent(ref) : ''}`.replace( + /\/$/, + '' + ) + }; +} + +export function sourceRepositoryKey(source) { + const parsed = typeof source === 'string' ? parseSourceOperand(source) : source; + return `${parsed.type}:${parsed.owner}/${parsed.repo}`; +} + +export function replaceSourceRef(input, newRef) { + if (!newRef || hasGlobMagic(newRef)) { + throw new Error('A replacement ref must be exact and non-empty'); + } + + const parsed = parseSourceOperand(input); + const qualifierText = formatQualifiers(parsed.qualifiers); + const subpathText = input.includes('#') ? input.slice(input.indexOf('#')) : ''; + return `pkg:github/${encodeURIComponent(parsed.owner)}/${encodeURIComponent(parsed.repo)}@${encodeURIComponent(newRef)}${qualifierText}${subpathText}`; +} + +function decodePart(value, label) { + try { + return decodeURIComponent(value); + } catch { + throw new Error(`Malformed percent-encoding in PURL ${label}`); + } +} + +function decodeSubpath(rawSubpath) { + if (!rawSubpath) { + return ''; + } + + return rawSubpath + .split('/') + .map(segment => decodePart(segment, 'subpath')) + .join('/'); +} + +function validateSubpath(subpath, rawSubpath) { + if (!subpath) { + return; + } + + if (subpath.startsWith('/') || subpath.startsWith('\\')) { + throw new Error('A PURL subpath must be relative to the repository root'); + } + + if (/%2f|%5c/i.test(rawSubpath)) { + throw new Error('A PURL subpath segment must not contain an encoded slash'); + } + + const segments = subpath.split('/'); + if (segments.some(segment => segment === '.' || segment === '..' || segment === '')) { + throw new Error('A PURL subpath must not contain empty, ".", or ".." segments'); + } + + if (segments.some(segment => segment.includes('\\') || segment.includes('\0'))) { + throw new Error('A PURL subpath contains an unsafe path character'); + } +} + +function parseQualifiers(raw) { + if (!raw) { + return []; + } + + return raw.split('&').map(pair => { + const separator = pair.indexOf('='); + if (separator < 1) { + throw new Error('Malformed PURL qualifier; expected key=value'); + } + + return [ + decodePart(pair.slice(0, separator), 'qualifier'), + decodePart(pair.slice(separator + 1), 'qualifier') + ]; + }); +} + +function formatQualifiers(qualifiers) { + if (qualifiers.length === 0) { + return ''; + } + + return `?${qualifiers + .map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`) + .join('&')}`; +} + +function hasGlobMagic(value) { + return new Minimatch(value, { magicalBraces: true }).hasMagic(); +} diff --git a/src/transfer-rules.js b/src/transfer-rules.js new file mode 100644 index 0000000..95fa62a --- /dev/null +++ b/src/transfer-rules.js @@ -0,0 +1,56 @@ +import { Minimatch } from 'minimatch'; +import { PRESETS } from './presets.js'; + +export function compileTransferRules({ globs = [], iglobs = [], preset = null } = {}) { + if (preset && !PRESETS[preset]) { + throw new Error(`Unknown preset "${preset}"`); + } + + const definitions = [ + ...(preset ? PRESETS[preset].filters.map(pattern => ({ pattern, nocase: false })) : []), + ...globs.map(pattern => ({ pattern, nocase: false })), + ...iglobs.map(pattern => ({ pattern, nocase: true })) + ]; + const rules = definitions.map(compileRule); + const defaultIncluded = !rules.some(rule => rule.include); + + return { + active: rules.length > 0, + globs: [...globs], + iglobs: [...iglobs], + preset, + matches(path) { + const normalized = path.replaceAll('\\', '/'); + let included = defaultIncluded; + for (const rule of rules) { + if (rule.matcher.match(normalized)) { + included = rule.include; + } + } + + return included; + } + }; +} + +function compileRule({ pattern, nocase }) { + if (typeof pattern !== 'string' || pattern.length === 0) { + throw new Error('A transfer glob must not be empty'); + } + + const include = !pattern.startsWith('!'); + const expression = include ? pattern : pattern.slice(1); + if (!expression) { + throw new Error('A transfer glob must not be empty'); + } + + return { + include, + matcher: new Minimatch(expression, { + dot: true, + magicalBraces: true, + nocase, + nonegate: true + }) + }; +} diff --git a/src/vlurpfile.js b/src/vlurpfile.js index 14464c5..1503c0a 100644 --- a/src/vlurpfile.js +++ b/src/vlurpfile.js @@ -1,28 +1,16 @@ import { resolve } from 'node:path'; import { PRESETS } from './presets.js'; +import { parseSourceOperand, replaceSourceRef } from './source-operand.js'; -/** - * Parses a .vlurpfile and returns an array of vlurp commands. - * - * Format: - * # Comments start with # - * vlurp user/repo -d ./vlurp - * vlurp user/repo -d ./vlurp --filter "pattern" - * vlurp user/repo --preset claude - */ +/** Parse legacy and cp-style commands from a .vlurpfile. */ export function parseVlurpfile(content) { - const lines = content.split('\n'); const entries = []; - - for (const line of lines) { + for (const line of content.split('\n')) { const trimmed = line.trim(); - - // Skip empty lines and comments if (!trimmed || trimmed.startsWith('#')) { continue; } - // Parse the vlurp command const entry = parseVlurpLine(trimmed); if (entry) { entries.push(entry); @@ -32,109 +20,94 @@ export function parseVlurpfile(content) { return entries; } -/** - * Update the --ref value for a specific source in vlurpfile content. - * Preserves comments, blank lines, and argument ordering. - * If the entry has no --ref, appends one. - * Returns the updated content string. - */ export function updateRef(content, source, newRef) { - const lines = content.split('\n'); - const result = []; - - for (const line of lines) { - const trimmed = line.trim(); - - // Preserve comments and blank lines as-is - if (!trimmed || trimmed.startsWith('#')) { - result.push(line); - continue; - } - - // Check if this line matches the target source - const lineSource = extractSource(trimmed); - if (lineSource !== source) { - result.push(line); - continue; - } - - // This line matches -- update or insert --ref - result.push(replaceRef(line, newRef)); - } - - return result.join('\n'); + return updateRefs(content, new Map([[source, newRef]])); } /** - * Update --ref values for multiple sources at once. - * `updates` is a Map or object of { source: newRef }. - * Returns the updated content string. + * Update refs without reserializing command lines. PURL refs are embedded in + * each source operand; legacy entries continue to use --ref. */ export function updateRefs(content, updates) { const map = updates instanceof Map ? updates : new Map(Object.entries(updates)); - const lines = content.split('\n'); - const result = []; - - for (const line of lines) { - const trimmed = line.trim(); - - if (!trimmed || trimmed.startsWith('#')) { - result.push(line); - continue; - } - - const lineSource = extractSource(trimmed); - if (lineSource && map.has(lineSource)) { - result.push(replaceRef(line, map.get(lineSource))); - } else { - result.push(line); - } - } - - return result.join('\n'); + return content + .split('\n') + .map(line => updateLineRefs(line, map)) + .join('\n'); } -/** - * Extract the source (e.g. "user/repo") from a vlurpfile line. - */ -function extractSource(line) { +export function parseVlurpLine(line) { const command = line.startsWith('vlurp ') ? line.slice(6).trim() : line; const args = parseArgs(command); - return args.length > 0 ? args[0] : null; -} + if (args.length === 0) { + return null; + } -/** - * Replace or insert --ref in a single vlurpfile line. - */ -function replaceRef(line, newRef) { - // Match existing --ref and its value (handles quoted and unquoted) - const refPattern = /--ref\s+(?:"[^"]*"|\S+)/; - if (refPattern.test(line)) { - return line.replace(refPattern, `--ref ${newRef}`); + if (args[0].startsWith('pkg:')) { + return parseCopyEntry(args); } - // No existing --ref -- append before trailing newline/whitespace - const trimmed = line.trimEnd(); - return `${trimmed} --ref ${newRef}`; + return parseLegacyEntry(args); } -/** - * Parses a single vlurp command line. - */ -function parseVlurpLine(line) { - // Remove 'vlurp' prefix if present - const command = line.startsWith('vlurp ') ? line.slice(6).trim() : line; +function parseCopyEntry(args) { + const sources = []; + let index = 0; + while (args[index]?.startsWith('pkg:')) { + parseSourceOperand(args[index]); + sources.push(args[index++]); + } - // Simple argument parser - const args = parseArgs(command); + const destination = args[index++]; + if (!destination || destination.startsWith('-')) { + throw new Error('A cp-style vlurpfile entry requires SOURCE... DEST'); + } - if (args.length === 0) { - return null; + const entry = { + mode: 'copy', + source: sources[0], + sources, + destination, + targetPath: resolve(destination), + rootDir: resolve(destination), + filters: [], + globs: [], + iglobs: [], + preset: null, + ref: parseSourceOperand(sources[0]).ref, + as: null, + force: false + }; + + for (; index < args.length; index++) { + const arg = args[index]; + if (arg === '-f' || arg === '--force') { + entry.force = true; + } else if (arg === '--glob' && args[index + 1]) { + entry.globs.push(args[++index]); + } else if (arg === '--iglob' && args[index + 1]) { + entry.iglobs.push(args[++index]); + } else if (arg === '--preset' && args[index + 1]) { + entry.preset = args[++index]; + } else if (arg === '-n' || arg === '--dry-run' || arg === '-q' || arg === '--quiet') { + // Invocation-level presentation flags are valid but not persisted on the entry. + } else { + throw new Error(`Unsupported option in cp-style vlurpfile entry: ${arg}`); + } } + assertKnownPreset(entry.preset); + + return entry; +} + +function parseLegacyEntry(args) { const source = args[0]; const entry = { + mode: 'legacy', source, + sources: [source], + destination: null, rootDir: null, filters: [], preset: null, @@ -143,37 +116,35 @@ function parseVlurpLine(line) { force: false }; - // Parse options - for (let i = 1; i < args.length; i++) { - const arg = args[i]; - - if (arg === '-d' && args[i + 1]) { - entry.rootDir = args[++i]; - } else if (arg === '--filter' && args[i + 1]) { - entry.filters.push(args[++i]); - } else if (arg === '--preset' && args[i + 1]) { - entry.preset = args[++i]; + for (let index = 1; index < args.length; index++) { + const arg = args[index]; + if (arg === '-d' && args[index + 1]) { + entry.rootDir = args[++index]; + } else if (arg === '--filter' && args[index + 1]) { + entry.filters.push(args[++index]); + } else if (arg === '--preset' && args[index + 1]) { + entry.preset = args[++index]; if (PRESETS[entry.preset]) { entry.filters = [...PRESETS[entry.preset].filters]; } - } else if (arg === '--ref' && args[i + 1]) { - entry.ref = args[++i]; - } else if (arg === '--as' && args[i + 1]) { - entry.as = args[++i]; + } else if (arg === '--ref' && args[index + 1]) { + entry.ref = args[++index]; + } else if (arg === '--as' && args[index + 1]) { + entry.as = args[++index]; } else if (arg === '-f' || arg === '--force') { entry.force = true; } } - // Calculate target path + assertKnownPreset(entry.preset); + if (entry.as && entry.rootDir) { entry.targetPath = resolve(entry.rootDir, entry.as); } else if (entry.as) { entry.targetPath = resolve(entry.as); } else if (entry.rootDir) { - const parts = source.split('/'); - if (parts.length >= 2) { - const [user, repo] = parts; + const [user, repo] = source.split('/'); + if (user && repo) { entry.targetPath = resolve(entry.rootDir, user, repo); } } @@ -181,17 +152,107 @@ function parseVlurpLine(line) { return entry; } -/** - * Simple argument parser that handles quoted strings. - */ -function parseArgs(string_) { +function assertKnownPreset(preset) { + if (preset && !PRESETS[preset]) { + throw new Error(`Unknown preset "${preset}"`); + } +} + +function updateLineRefs(line, updates) { + const trimmed = line.trim(); + if (!trimmed || trimmed.startsWith('#')) { + return line; + } + + let entry; + try { + entry = parseVlurpLine(trimmed); + } catch { + return line; + } + + if (entry.mode === 'copy') { + let updated = line; + for (const source of entry.sources) { + if (updates.has(source)) { + updated = updated.replace(source, replaceSourceRef(source, updates.get(source))); + } + } + + return updated; + } + + if (!updates.has(entry.source)) { + return line; + } + + return replaceLegacyRef(line, updates.get(entry.source)); +} + +function replaceLegacyRef(line, newRef) { + const refPattern = /--ref\s+(?:"[^"]*"|'[^']*'|\S+)/; + if (refPattern.test(line)) { + return line.replace(refPattern, `--ref ${newRef}`); + } + + return `${line.trimEnd()} --ref ${newRef}`; +} + +/** A small shell-word parser: quoting groups text, but nothing is expanded. */ +export function parseArgs(string_) { const args = []; - const regex = /(?:[^\s"]+|"[^"]*")+/g; - let match; + let current = ''; + let quote = null; + let escaped = false; + + for (const character of string_) { + if (escaped) { + current += character; + escaped = false; + continue; + } + + if (character === '\\' && quote !== "'") { + escaped = true; + continue; + } + + if (quote) { + if (character === quote) { + quote = null; + } else { + current += character; + } + + continue; + } + + if (character === '#' && current === '') { + break; + } + + if (character === '"' || character === "'") { + quote = character; + } else if (/\s/.test(character)) { + if (current) { + args.push(current); + current = ''; + } + } else { + current += character; + } + } + + if (quote) { + throw new Error('Unterminated quote in vlurpfile entry'); + } + + if (escaped) { + current += '\\'; + } - while ((match = regex.exec(string_)) !== null) { - // Remove surrounding quotes - args.push(match[0].replaceAll(/^"|"$/g, '')); + if (current) { + args.push(current); } return args; diff --git a/test/cli.test.js b/test/cli.test.js index fdb2317..6b27848 100644 --- a/test/cli.test.js +++ b/test/cli.test.js @@ -12,6 +12,19 @@ describe('CLI', () => { const output = execSync(`node ${binPath} --help`, { encoding: 'utf8' }); assert.ok(output.includes('vlurp'), 'Help should mention vlurp'); assert.ok(output.includes('--help'), 'Help should show --help option'); + assert.ok(output.includes('SOURCE... DEST'), 'Help should describe cp-style operands'); + assert.ok(output.includes('pkg:github/'), 'Help should describe PURL sources'); + assert.ok(output.includes('Presets:'), 'Help should keep presets as a first-class mode'); + assert.ok(output.includes('--preset'), 'Help should document the preset option'); + assert.ok(output.includes('--glob'), 'Help should document case-sensitive transfer patterns'); + assert.ok( + output.includes('--iglob'), + 'Help should document case-insensitive transfer patterns' + ); + assert.ok( + output.indexOf('vlurp /') < output.indexOf("vlurp 'pkg:github/"), + 'Help should present repository shorthand before precise PURL copies' + ); }); it('should show usage when called without arguments', () => { @@ -23,4 +36,33 @@ describe('CLI', () => { assert.ok(err.status !== 0, 'Should exit with non-zero status'); } }); + + it('requires a destination for a PURL source', () => { + assert.throws( + () => execSync(`node ${binPath} pkg:github/user/repo`, { encoding: 'utf8', stdio: 'pipe' }), + error => error.status !== 0 && error.stderr.includes('requires SOURCE and DEST') + ); + }); + + it('keeps repository selection flags separate from PURL operands', () => { + assert.throws( + () => + execSync(`node ${binPath} pkg:github/user/repo#docs ./docs --filter '*.md'`, { + encoding: 'utf8', + stdio: 'pipe' + }), + error => error.status !== 0 && error.stderr.includes('cannot be used with PURL sources') + ); + }); + + it('rejects an unknown preset before resolving a remote source', () => { + assert.throws( + () => + execSync(`node ${binPath} pkg:github/user/repo ./dest --preset made-up`, { + encoding: 'utf8', + stdio: 'pipe' + }), + error => error.status !== 0 && error.stderr.includes('unknown preset') + ); + }); }); diff --git a/test/copy-plan.test.js b/test/copy-plan.test.js new file mode 100644 index 0000000..68cae45 --- /dev/null +++ b/test/copy-plan.test.js @@ -0,0 +1,240 @@ +import { strict as assert } from 'node:assert'; +import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import { afterEach, beforeEach, describe, it } from 'node:test'; +import { createCopyPlan, executeCopyPlan, selectRepositoryEntries } from '../src/copy-plan.js'; + +describe('cp-style copy planning', () => { + let temporaryDirectory; + let repository; + + beforeEach(async () => { + temporaryDirectory = await mkdtemp(join(tmpdir(), 'vlurp-copy-plan-')); + repository = join(temporaryDirectory, 'repo'); + await mkdir(join(repository, 'skills', 'writing-a'), { recursive: true }); + await mkdir(join(repository, 'skills', 'writing-b'), { recursive: true }); + await writeFile(join(repository, 'skills', 'writing-a', 'SKILL.md'), 'a'); + await writeFile(join(repository, 'skills', 'writing-b', 'SKILL.md'), 'b'); + await writeFile(join(repository, 'README.md'), 'readme'); + }); + + afterEach(async () => { + await rm(temporaryDirectory, { recursive: true, force: true }); + }); + + it('selects glob matches as independent cp sources', async () => { + const matches = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: 'skills/writing-*', + pattern: true + }); + + assert.deepEqual( + matches.map(match => match.relativePath), + ['skills/writing-a', 'skills/writing-b'] + ); + }); + + it('copies matched basenames into an existing destination directory', async () => { + const destination = join(temporaryDirectory, 'destination'); + await mkdir(destination); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: 'skills/writing-*', + pattern: true, + raw: 'pkg:github/user/repo#skills/writing-*' + }); + const plan = await createCopyPlan(selections, destination); + + assert.deepEqual( + plan.entries.map(entry => entry.targetPath), + [join(destination, 'writing-a'), join(destination, 'writing-b')] + ); + + await executeCopyPlan(plan); + assert.equal(await readFile(join(destination, 'writing-a', 'SKILL.md'), 'utf8'), 'a'); + assert.equal(await readFile(join(destination, 'writing-b', 'SKILL.md'), 'utf8'), 'b'); + }); + + it('filters a cp source with case-sensitive glob rules', async () => { + const destination = join(temporaryDirectory, 'destination'); + await mkdir(destination); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: '', + pattern: false, + raw: 'pkg:github/user/repo' + }); + const plan = await createCopyPlan(selections, destination, { + globs: ['**/*.md', '!README.md'] + }); + + await executeCopyPlan(plan); + assert.equal( + await readFile(join(destination, 'repo', 'skills', 'writing-a', 'SKILL.md'), 'utf8'), + 'a' + ); + await assert.rejects(readFile(join(destination, 'repo', 'README.md')), /ENOENT/); + }); + + it('filters a cp source with case-insensitive iglob rules', async () => { + const destination = join(temporaryDirectory, 'destination'); + await mkdir(destination); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: 'skills/writing-a', + pattern: false, + raw: 'pkg:github/user/repo#skills/writing-a' + }); + const plan = await createCopyPlan(selections, destination, { + iglobs: ['**/skill.md'] + }); + + await executeCopyPlan(plan); + assert.equal(await readFile(join(destination, 'writing-a', 'SKILL.md'), 'utf8'), 'a'); + }); + + it('omits selected directories with no matching transfer files', async () => { + const destination = join(temporaryDirectory, 'destination'); + await mkdir(destination); + await writeFile(join(repository, 'skills', 'writing-a', 'ONLY.txt'), 'selected'); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: 'skills/writing-*', + pattern: true, + raw: 'pkg:github/user/repo#skills/writing-*' + }); + const plan = await createCopyPlan(selections, destination, { globs: ['ONLY.txt'] }); + + assert.deepEqual( + plan.entries.map(entry => entry.basename), + ['writing-a'] + ); + await executeCopyPlan(plan); + assert.equal(await readFile(join(destination, 'writing-a', 'ONLY.txt'), 'utf8'), 'selected'); + await assert.rejects(readFile(join(destination, 'writing-b', 'SKILL.md')), /ENOENT/); + }); + + it('allows transfer rules to exclude repository lineage metadata', async () => { + const destination = join(temporaryDirectory, 'destination'); + await writeFile(join(repository, '.vlurp.jsonl'), 'remote metadata'); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: '', + pattern: false, + raw: 'pkg:github/user/repo' + }); + const plan = await createCopyPlan(selections, destination, { globs: ['**/*.md'] }); + + await executeCopyPlan(plan); + assert.equal(await readFile(join(destination, 'README.md'), 'utf8'), 'readme'); + }); + + it('rejects symbolic links in filtered destination paths before copying', async () => { + const destination = join(temporaryDirectory, 'destination'); + const outside = join(temporaryDirectory, 'outside'); + const sourceDirectory = join(repository, 'skills', 'writing-a'); + await mkdir(join(sourceDirectory, 'nested')); + await writeFile(join(sourceDirectory, 'nested', 'guide.md'), 'safe'); + await mkdir(join(destination, 'writing-a'), { recursive: true }); + await mkdir(outside); + await symlink(outside, join(destination, 'writing-a', 'nested')); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: 'skills/writing-a', + pattern: false, + raw: 'pkg:github/user/repo#skills/writing-a' + }); + + await assert.rejects( + createCopyPlan(selections, destination, { globs: ['**/*.md'] }), + /symbolic link/i + ); + }); + + it('makes a missing destination the copied file itself', async () => { + const destination = join(temporaryDirectory, 'renamed.md'); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: 'README.md', + pattern: false, + raw: 'pkg:github/user/repo#README.md' + }); + const plan = await createCopyPlan(selections, destination); + + assert.equal(plan.entries[0].targetPath, destination); + }); + + it('requires an existing directory for multiple cp sources', async () => { + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: 'skills/writing-*', + pattern: true, + raw: 'pkg:github/user/repo#skills/writing-*' + }); + + await assert.rejects( + createCopyPlan(selections, join(temporaryDirectory, 'missing')), + /existing directory/i + ); + }); + + it('rejects basename collisions before copying', async () => { + const destination = join(temporaryDirectory, 'destination'); + await mkdir(destination); + const source = join(repository, 'README.md'); + const selection = { + sourcePath: source, + relativePath: 'README.md', + basename: 'README.md', + kind: 'file' + }; + + await assert.rejects(createCopyPlan([selection, selection], destination), /collision/i); + }); + + it('rejects nested symbolic links before copying anything', async () => { + const destination = join(temporaryDirectory, 'destination'); + const sourceDirectory = join(repository, 'skills', 'writing-a'); + await symlink(join(repository, 'README.md'), join(sourceDirectory, 'linked.md')); + const selections = [ + { + sourcePath: sourceDirectory, + relativePath: 'skills/writing-a', + basename: 'writing-a', + kind: 'directory' + } + ]; + + await assert.rejects(createCopyPlan(selections, destination), /symbolic link/i); + await assert.rejects(readFile(destination), /ENOENT/); + }); + + it('does not let remote content replace lineage metadata', async () => { + await writeFile(join(repository, '.vlurp.jsonl'), '{"forged":true}\n'); + const selections = await selectRepositoryEntries(repository, { + repo: 'repo', + subpath: '', + pattern: false, + raw: 'pkg:github/user/repo' + }); + + await assert.rejects( + createCopyPlan(selections, join(temporaryDirectory, 'renamed-repo')), + /reserved vlurp metadata/i + ); + }); + + it('never resolves a selected path outside the materialized repository', async () => { + await assert.rejects( + selectRepositoryEntries(repository, { + repo: 'repo', + subpath: '../secret', + pattern: false + }), + /unsafe/i + ); + assert.equal(resolve(repository, '../secret').startsWith(`${resolve(repository)}/`), false); + }); +}); diff --git a/test/copy-sources.test.js b/test/copy-sources.test.js new file mode 100644 index 0000000..b894842 --- /dev/null +++ b/test/copy-sources.test.js @@ -0,0 +1,129 @@ +import { strict as assert } from 'node:assert'; +import { cp, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, beforeEach, describe, it } from 'node:test'; +import { copySources } from '../src/copy-sources.js'; +import { readLineage, verifyFiles } from '../src/lineage.js'; + +describe('copySources', () => { + let temporaryDirectory; + let fixtureRepository; + let destination; + let fetcher; + + beforeEach(async () => { + temporaryDirectory = await mkdtemp(join(tmpdir(), 'vlurp-copy-sources-')); + fixtureRepository = join(temporaryDirectory, 'fixture'); + destination = join(temporaryDirectory, 'skills'); + await mkdir(join(fixtureRepository, 'skills', 'writing-a'), { recursive: true }); + await mkdir(join(fixtureRepository, 'skills', 'writing-b'), { recursive: true }); + await mkdir(destination); + await writeFile(join(fixtureRepository, 'skills', 'writing-a', 'SKILL.md'), 'a'); + await writeFile(join(fixtureRepository, 'skills', 'writing-b', 'SKILL.md'), 'b'); + await writeFile(join(fixtureRepository, 'README.md'), 'boilerplate'); + + fetcher = { + async materialize() { + const materialized = await mkdtemp(join(tmpdir(), 'vlurp-materialized-')); + await cp(fixtureRepository, materialized, { recursive: true }); + return { + path: materialized, + cleanup: () => rm(materialized, { recursive: true, force: true }) + }; + } + }; + }); + + afterEach(async () => { + await rm(temporaryDirectory, { recursive: true, force: true }); + }); + + it('copies glob matches with basename semantics and records verifiable lineage', async () => { + const result = await copySources({ + sources: ['pkg:github/user/repo@abc1234#skills/writing-*'], + destination, + force: true, + fetcher + }); + + assert.equal(result.status, 'complete'); + assert.equal(result.fileCount, 2); + assert.equal(await readFile(join(destination, 'writing-a', 'SKILL.md'), 'utf8'), 'a'); + assert.equal(await readFile(join(destination, 'writing-b', 'SKILL.md'), 'utf8'), 'b'); + + const records = await readLineage(join(destination, '.vlurp.jsonl')); + assert.equal(records.length, 1); + assert.equal(records[0].source, 'pkg:github/user/repo@abc1234#skills/writing-*'); + assert.deepEqual(Object.keys(records[0].files), ['writing-a/SKILL.md', 'writing-b/SKILL.md']); + const verification = await verifyFiles(destination, records); + assert.equal(verification.filter(result => result.status !== 'ok').length, 0); + }); + + it('does not write when an overwrite is declined', async () => { + await mkdir(join(destination, 'writing-a')); + await writeFile(join(destination, 'writing-a', 'SKILL.md'), 'local'); + + const result = await copySources({ + sources: ['pkg:github/user/repo#skills/writing-a'], + destination, + fetcher, + confirm: async () => false + }); + + assert.equal(result.status, 'cancelled'); + assert.equal(await readFile(join(destination, 'writing-a', 'SKILL.md'), 'utf8'), 'local'); + }); + + it('expands presets into transfer rules for PURL cp sources', async () => { + await writeFile(join(fixtureRepository, 'skills', 'writing-a', 'script.js'), 'ignored'); + await copySources({ + sources: ['pkg:github/user/repo#skills/writing-a'], + destination, + preset: 'skills', + force: true, + fetcher + }); + + assert.equal(await readFile(join(destination, 'writing-a', 'SKILL.md'), 'utf8'), 'a'); + await assert.rejects(readFile(join(destination, 'writing-a', 'script.js')), /ENOENT/); + }); + + it('places lineage inside a newly named directory copy', async () => { + const namedDestination = join(temporaryDirectory, 'named-skill'); + await copySources({ + sources: ['pkg:github/user/repo#skills/writing-a'], + destination: namedDestination, + force: true, + fetcher + }); + + const records = await readLineage(join(namedDestination, '.vlurp.jsonl')); + assert.equal(records.length, 1); + assert.deepEqual(Object.keys(records[0].files), ['SKILL.md']); + const verification = await verifyFiles(namedDestination, records); + assert.equal(verification.filter(result => result.status !== 'ok').length, 0); + }); + + it('reuses one materialization for selectors from the same repository and ref', async () => { + let calls = 0; + const countingFetcher = { + async materialize(...args) { + calls++; + return fetcher.materialize(...args); + } + }; + + await copySources({ + sources: [ + 'pkg:github/user/repo@abc#skills/writing-a', + 'pkg:github/user/repo@abc#skills/writing-b' + ], + destination, + force: true, + fetcher: countingFetcher + }); + + assert.equal(calls, 1); + }); +}); diff --git a/test/e2e/ci/interactive.test.js b/test/e2e/ci/interactive.test.js index f3408ce..5cee609 100644 --- a/test/e2e/ci/interactive.test.js +++ b/test/e2e/ci/interactive.test.js @@ -9,6 +9,30 @@ import { runVlurpInPty } from '../helpers/cli.js'; const describePty = process.platform === 'linux' ? describe : describe.skip; describePty('interactive CLI', () => { + it('prompts for a PURL destination collision and cancels semantically', async t => { + const workspace = await mkdtemp(join(tmpdir(), 'vlurp-purl-pty-e2e-')); + const destination = join(workspace, 'skills'); + const target = join(destination, 'writing-beats'); + const marker = join(target, 'keep.txt'); + t.after(() => rm(workspace, { recursive: true, force: true })); + + await mkdir(target, { recursive: true }); + await writeFile(marker, 'keep me'); + const source = + 'pkg:github/mattpocock/skills@2ab958093e83e0ec752e6c1c5932da465bf23e0c#skills/in-progress/writing-beats'; + const cli = runVlurpInPty([source, destination, '--quiet'], { cwd: workspace }); + + await cli.waitForText('Continue? (y/N):'); + assert.match(cli.output, /1 destination path\(s\) already exist/); + + cli.write('n\r'); + const exit = await cli.waitForExit(); + + assert.equal(exit.exitCode, 0, cli.transcript); + assert.match(cli.output, /vlurping cancelled/); + assert.equal(await readFile(marker, 'utf8'), 'keep me'); + }); + it('prompts before overwriting and preserves files when cancelled', async t => { const workspace = await mkdtemp(join(tmpdir(), 'vlurp-pty-e2e-')); const destination = join(workspace, 'downloads'); diff --git a/test/e2e/ci/purl-copy.test.js b/test/e2e/ci/purl-copy.test.js new file mode 100644 index 0000000..a345366 --- /dev/null +++ b/test/e2e/ci/purl-copy.test.js @@ -0,0 +1,50 @@ +import { strict as assert } from 'node:assert'; +import { mkdir, mkdtemp, readdir, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { describe, it } from 'node:test'; +import { runVlurp } from '../helpers/cli.js'; + +describe('PURL copy', () => { + it('copies live GitHub glob matches with cp basename semantics', async t => { + const workspace = await mkdtemp(join(tmpdir(), 'vlurp-purl-e2e-')); + const destination = join(workspace, 'skills'); + t.after(() => rm(workspace, { recursive: true, force: true })); + + await mkdir(destination); + const source = + 'pkg:github/mattpocock/skills@2ab958093e83e0ec752e6c1c5932da465bf23e0c#skills/in-progress/writing-*'; + const result = await runVlurp( + [ + source, + destination, + '--preset', + 'skills', + '--glob', + '!README.md', + '--iglob', + '**/skill.md', + '--force', + '--quiet' + ], + { cwd: workspace } + ); + + assert.equal( + result.code, + 0, + `vlurp exited unsuccessfully\nstdout:\n${result.stdout}\nstderr:\n${result.stderr}` + ); + assert.doesNotMatch(result.stdout, /Error:/, result.stdout); + assert.equal(result.stderr, ''); + assert.deepEqual((await readdir(destination)).sort(), [ + '.vlurp.jsonl', + 'writing-beats', + 'writing-fragments', + 'writing-shape' + ]); + for (const directory of ['writing-beats', 'writing-fragments', 'writing-shape']) { + assert.deepEqual(await readdir(join(destination, directory)), ['SKILL.md']); + } + }); +}); diff --git a/test/lineage.test.js b/test/lineage.test.js index 6f38110..2cb0b2c 100644 --- a/test/lineage.test.js +++ b/test/lineage.test.js @@ -185,6 +185,59 @@ describe('lineage', () => { const records = await readLineage(join(fixturesDir, 'nonexistent.jsonl')); assert.deepEqual(records, []); }); + + it('replaces cp-style lineage when only the embedded ref changes', async () => { + const jsonlPath = join(fixturesDir, '.vlurp.jsonl'); + const base = { + schema: 2, + lineage_key: 'github:user/repo#skills/a', + destination: '.', + files_root_relative: true, + files: {} + }; + await appendLineage(jsonlPath, { + ...base, + source: 'pkg:github/user/repo@old#skills/a', + ref: 'old' + }); + await appendLineage(jsonlPath, { + ...base, + source: 'pkg:github/user/repo@new#skills/a', + ref: 'new' + }); + + const records = await readLineage(jsonlPath); + assert.equal(records.length, 1); + assert.equal(records[0].ref, 'new'); + }); + + it('gives the latest cp-style source ownership of overwritten files', async () => { + const jsonlPath = join(fixturesDir, '.vlurp.jsonl'); + await appendLineage(jsonlPath, { + schema: 2, + lineage_key: 'github:user/first#docs', + source: 'pkg:github/user/first#docs', + destination: '.', + files_root_relative: true, + files: { + 'shared.md': { sha256: 'old', size: 1 }, + 'first-only.md': { sha256: 'first', size: 1 } + } + }); + await appendLineage(jsonlPath, { + schema: 2, + lineage_key: 'github:user/second#docs', + source: 'pkg:github/user/second#docs', + destination: '.', + files_root_relative: true, + files: { 'shared.md': { sha256: 'new', size: 1 } } + }); + + const records = await readLineage(jsonlPath); + assert.equal(records.length, 2); + assert.deepEqual(Object.keys(records[0].files), ['first-only.md']); + assert.deepEqual(Object.keys(records[1].files), ['shared.md']); + }); }); describe('verifyFiles', () => { diff --git a/test/manifest-sources.test.js b/test/manifest-sources.test.js new file mode 100644 index 0000000..a91f275 --- /dev/null +++ b/test/manifest-sources.test.js @@ -0,0 +1,32 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; +import { expandManifestSources, manifestSourceMatches } from '../src/manifest-sources.js'; +import { parseVlurpfile } from '../src/vlurpfile.js'; + +describe('manifest source cohesion', () => { + it('expands multiple PURLs into independently pinnable repository sources', () => { + const entries = parseVlurpfile( + 'vlurp pkg:github/user/one@abc#docs pkg:github/user/two#skills/* ./content/' + ); + const sources = expandManifestSources(entries); + + assert.deepEqual( + sources.map(source => [source.repositoryKey, source.ref]), + [ + ['github:user/one', 'abc'], + ['github:user/two', null] + ] + ); + assert.equal(manifestSourceMatches(sources[1], 'user/two'), true); + assert.equal(manifestSourceMatches(sources[1], 'github:user/two'), true); + }); + + it('keeps legacy vlurpfile entries available during migration', () => { + const sources = expandManifestSources( + parseVlurpfile('vlurp user/repo -d ./content --ref abc1234') + ); + + assert.equal(sources[0].repositoryKey, 'github:user/repo'); + assert.equal(sources[0].ref, 'abc1234'); + }); +}); diff --git a/test/source-operand.test.js b/test/source-operand.test.js new file mode 100644 index 0000000..6d1cc0d --- /dev/null +++ b/test/source-operand.test.js @@ -0,0 +1,66 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; +import { + parseSourceOperand, + replaceSourceRef, + sourceRepositoryKey +} from '../src/source-operand.js'; + +describe('PURL source operands', () => { + it('parses a GitHub PURL pattern', () => { + const source = parseSourceOperand( + 'pkg:github/mattpocock/skills@2ab9580#skills/in-progress/writing-*' + ); + + assert.equal(source.type, 'github'); + assert.equal(source.owner, 'mattpocock'); + assert.equal(source.repo, 'skills'); + assert.equal(source.ref, '2ab9580'); + assert.equal(source.subpath, 'skills/in-progress/writing-*'); + assert.equal(source.pattern, true); + assert.equal(sourceRepositoryKey(source), 'github:mattpocock/skills'); + }); + + it('accepts an unpinned repository root', () => { + const source = parseSourceOperand('pkg:github/mattpocock/skills'); + + assert.equal(source.ref, null); + assert.equal(source.subpath, ''); + assert.equal(source.pattern, false); + }); + + it('decodes exact subpath segments without treating encoded stars as patterns', () => { + const source = parseSourceOperand('pkg:github/user/repo#docs/literal%2Aname.md'); + + assert.equal(source.subpath, 'docs/literal*name.md'); + assert.equal(source.pattern, false); + }); + + it('replaces an embedded ref without changing the selector', () => { + assert.equal( + replaceSourceRef('pkg:github/user/repo@old#skills/*', 'abc1234'), + 'pkg:github/user/repo@abc1234#skills/*' + ); + assert.equal( + replaceSourceRef('pkg:github/user/repo#skills/*', 'abc1234'), + 'pkg:github/user/repo@abc1234#skills/*' + ); + }); + + for (const [input, message] of [ + ['pkg:npm/user/repo', 'Unsupported PURL type'], + ['pkg:github/user', 'owner and repository'], + ['pkg:github/user/repo@feature/*#docs', 'Glob syntax is only allowed'], + ['pkg:github/user/repo@#docs', 'must not be empty'], + ['pkg:github/user/repo@feature%2A#docs', 'Glob syntax is only allowed'], + ['pkg:github/us*/repo#docs', 'Glob syntax is only allowed'], + ['pkg:github/user/repo#/absolute', 'relative'], + ['pkg:github/user/repo#docs/../secret', 'must not contain'], + ['pkg:github/user/repo#docs/%2Fsecret', 'encoded slash'], + ['github:user/repo', 'PURL'] + ]) { + it(`rejects unsafe or unsupported source: ${input}`, () => { + assert.throws(() => parseSourceOperand(input), new RegExp(message, 'i')); + }); + } +}); diff --git a/test/transfer-rules.test.js b/test/transfer-rules.test.js new file mode 100644 index 0000000..f66f33c --- /dev/null +++ b/test/transfer-rules.test.js @@ -0,0 +1,46 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; +import { compileTransferRules } from '../src/transfer-rules.js'; + +describe('transfer glob rules', () => { + it('uses positive globs as an allowlist and supports exclusions', () => { + const rules = compileTransferRules({ + globs: ['**/*.md', '!README.md'] + }); + + assert.equal(rules.matches('skills/example/SKILL.md'), true); + assert.equal(rules.matches('README.md'), false); + assert.equal(rules.matches('skills/example/script.js'), false); + }); + + it('makes iglob rules case-insensitive', () => { + const sensitive = compileTransferRules({ globs: ['**/skill.md'] }); + const insensitive = compileTransferRules({ iglobs: ['**/skill.md'] }); + + assert.equal(sensitive.matches('skills/example/SKILL.md'), false); + assert.equal(insensitive.matches('skills/example/SKILL.md'), true); + }); + + it('lets later explicit rules override preset rules', () => { + const rules = compileTransferRules({ + preset: 'skills', + globs: ['README.md'] + }); + + assert.equal(rules.matches('README.md'), true); + assert.equal(rules.matches('LICENSE'), false); + assert.equal(rules.matches('skills/example/SKILL.md'), true); + }); + + it('includes unmatched files when every rule is an exclusion', () => { + const rules = compileTransferRules({ globs: ['!**/*.tmp'] }); + + assert.equal(rules.matches('src/index.js'), true); + assert.equal(rules.matches('cache/result.tmp'), false); + }); + + it('rejects unknown presets and empty patterns', () => { + assert.throws(() => compileTransferRules({ preset: 'missing' }), /unknown preset/i); + assert.throws(() => compileTransferRules({ globs: [''] }), /must not be empty/i); + }); +}); diff --git a/test/upgrade.test.js b/test/upgrade.test.js index 0b914da..4af3fff 100644 --- a/test/upgrade.test.js +++ b/test/upgrade.test.js @@ -206,6 +206,40 @@ Use Bash tool to run commands. assert.ok(catalog.generated_at); }); + it('should build catalog from cp-style root-relative lineage', async () => { + const skillDir = join(tempDir, 'writing-a'); + await mkdir(skillDir, { recursive: true }); + await writeFile( + join(skillDir, 'SKILL.md'), + `--- +name: writing-a +version: 2.0.0 +description: Root-relative skill +--- + +# Writing A +` + ); + await writeFile( + join(tempDir, '.vlurp.jsonl'), + `${JSON.stringify({ + schema: 2, + source: 'pkg:github/user/repo@abc1234#skills/writing-a', + ref: 'abc1234', + fetched_at: '2026-08-01T00:00:00Z', + files_root_relative: true, + files: { 'writing-a/SKILL.md': { sha256: 'fake', size: 100 } } + })}\n` + ); + + const catalog = await buildCatalog(tempDir); + assert.equal(catalog.skills['writing-a'].path, 'writing-a/SKILL.md'); + assert.equal( + catalog.skills['writing-a'].source, + 'pkg:github/user/repo@abc1234#skills/writing-a' + ); + }); + it('should diff catalogs from pre/post upgrade snapshots', async () => { // Simulate pre-upgrade catalog const preCatalog = { diff --git a/test/vlurpfile.test.js b/test/vlurpfile.test.js index f2edec1..c024305 100644 --- a/test/vlurpfile.test.js +++ b/test/vlurpfile.test.js @@ -3,6 +3,55 @@ import { describe, it } from 'node:test'; import { parseVlurpfile, updateRef, updateRefs } from '../src/vlurpfile.js'; describe('vlurpfile parser', () => { + it('parses quoted PURL sources and a cp destination', () => { + const content = + "vlurp 'pkg:github/mattpocock/skills@abc1234#skills/writing-*' './my skills/' --force"; + const [entry] = parseVlurpfile(content); + + assert.equal(entry.mode, 'copy'); + assert.deepEqual(entry.sources, ['pkg:github/mattpocock/skills@abc1234#skills/writing-*']); + assert.equal(entry.destination, './my skills/'); + assert.equal(entry.ref, 'abc1234'); + assert.equal(entry.force, true); + }); + + it('parses multiple PURL source operands', () => { + const [entry] = parseVlurpfile( + 'vlurp pkg:github/user/a#one pkg:github/user/b#two ./destination' + ); + + assert.deepEqual(entry.sources, ['pkg:github/user/a#one', 'pkg:github/user/b#two']); + assert.equal(entry.destination, './destination'); + }); + + it('parses glob, iglob, and preset transfer rules for PURL copies', () => { + const [entry] = parseVlurpfile( + "vlurp pkg:github/user/repo ./dest --preset skills --glob '!README.md' --iglob '**/skill.md'" + ); + + assert.equal(entry.preset, 'skills'); + assert.deepEqual(entry.globs, ['!README.md']); + assert.deepEqual(entry.iglobs, ['**/skill.md']); + }); + + it('rejects unknown presets', () => { + assert.throws( + () => parseVlurpfile('vlurp pkg:github/user/repo ./dest --preset made-up'), + /Unknown preset/ + ); + assert.throws(() => parseVlurpfile('vlurp user/repo --preset made-up'), /Unknown preset/); + }); + + it('distinguishes inline comments from PURL subpaths', () => { + const [entry] = parseVlurpfile( + 'vlurp pkg:github/user/repo@abc1234#skills/* ./skills/ --force # reviewed source' + ); + + assert.equal(entry.sources[0], 'pkg:github/user/repo@abc1234#skills/*'); + assert.equal(entry.destination, './skills/'); + assert.equal(entry.force, true); + }); + it('should parse --ref flag', () => { const content = 'vlurp dcramer/dex -d ./skills --ref 939f6cb'; const entries = parseVlurpfile(content); @@ -69,6 +118,24 @@ vlurp user/repo -d ./vlurp }); describe('vlurpfile writer - updateRef', () => { + it('updates a ref embedded in a PURL while preserving quotes and subpath', () => { + const source = 'pkg:github/user/repo@old1234#skills/writing-*'; + const content = `vlurp '${source}' ./skills/`; + + assert.equal( + updateRef(content, source, 'new5678'), + "vlurp 'pkg:github/user/repo@new5678#skills/writing-*' ./skills/" + ); + }); + + it('adds an embedded ref to an unpinned PURL', () => { + const source = 'pkg:github/user/repo#skills/a'; + assert.equal( + updateRef(`vlurp ${source} ./skills/`, source, 'abc1234'), + 'vlurp pkg:github/user/repo@abc1234#skills/a ./skills/' + ); + }); + it('should update an existing --ref value', () => { const content = 'vlurp user/repo -d ./skills --ref abc1234'; const result = updateRef(content, 'user/repo', 'def5678');