Conversation
Retarget sync-core/sync-plugin from this repo's V2 certified-assets canister to the legacy `assetstorage` canister shipped with dfx 0.32.0 (`ic-certified-assets`, api_version 2). Per-asset metadata now comes from `.ic-assets.json5` instead of `_headers`/`_redirects`. sync-core: - new `config.rs`: full `.ic-assets.json5`/`.ic-assets.json` parser ported from ic-asset (nested inheritance, glob `match`, `ignore`, `headers`, `cache.max_age`, `enable_aliasing`, `allow_raw_access`, `encodings`, `security_policy`) - new `security_policy.rs`: standard/hardened/disabled CSP presets - `scan.rs` rewritten as gather_asset_descriptors (walkdir + config-driven ignore; `.well-known` kept; config files excluded) - `canister.rs`: add `enable_aliasing` (CreateAsset) and `is_aliased` (SetAssetProperties + AssetProperties); drop RedirectRule/RulePattern/ SetRedirectRules/get_redirect_rules - `sync.rs`: drive CreateAsset/SetAssetProperties from AssetConfig; drop redirect/html-handling; keep chunk packing + commit staging - delete redirects.rs, headers.rs, html_handling.rs, glob.rs - update_properties ignores the canister-injected `Set-Cookie: ic_env=...` header (added by icp-cli's env-var step) so syncs stay idempotent e2e: - build.rs fetches the real 0.32.0 assetstorage.wasm.gz (sha256-pinned, installed gzipped, no decompress) instead of building this repo's canister - convert fixtures/tests to `.ic-assets.json5`; remove redirect/header/ multi-dir cases; add config.rs end-to-end test Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch to `sync_plugin` without changing the canister wasm. ### Context This swaps the asset-canister recipe's sync step from the built-in `type: assets` mechanism to `type: plugin`, **without changing the canister wasm** — it remains the legacy dfx `assetstorage` canister. The goal is to ease the upgrade path for existing asset-canister users once `type: assets` sync is dropped in the upcoming icp-cli release: their canister stays the same, only the sync mechanism changes. The plugin wasm is pinned to a pre-release hosted on the certified-assets repo: - URL: `https://github.com/dfinity/certified-assets/releases/download/migration-v2.2.0-209d688/sync_plugin.wasm` - sha256: `297c2ef05680d47ac70688d6cebed9bc3a41b2302f400739f894f4f413e6a5ee` That plugin is built from the **`migration`** branch of `dfinity/certified-assets`, a permanently separate branch (never merged into `main`) dedicated to producing a sync plugin that targets the *legacy* assetstorage canister and reads `.ic-assets.json5` config: - Branch: https://github.com/dfinity/certified-assets/tree/migration - Plugin source / rationale: dfinity/certified-assets#71 - Release hosting this wasm: https://github.com/dfinity/certified-assets/releases/tag/migration-v2.2.0-209d688 The `v2.2.0` in the tag tracks this recipe version (current latest is `asset-canister-v2.1.0`); the `-209d688` suffix pins the exact commit the wasm was built from. Future recipe bumps will get a matching `migration-vX.Y.Z-<hash>` release. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The sync plugin's scan step called `Path::canonicalize` on the manifest `dir`. On wasm32-wasip2 that calls `realpath`, which returns ENOENT for any path under a preopen whose guest name has more than one component (e.g. `src/frontend/dist`), even though ordinary access works. Nested `dir`s therefore failed with "canonicalize src/frontend/dist: No such file or directory"; single-component dirs like `dist` happened to work. Build the scan root by prepending `/` to the (host-validated, relative, `..`-free) manifest dir instead. This matches the shape canonicalize produced for single-component dirs and satisfies the absolute-root requirement of AssetSourceDirectoryConfiguration::load, while using only plain wasi:filesystem ops the preopen supports. Adds an e2e fixture + test using a nested `dirs` entry, which reproduces the reported failure without the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Pushed a fix for a sync failure reported on the forum (#97): a user with Root cause. Fix. Drop Test. Added an e2e fixture + |
PR #71 (migration) is usually in a conflicting/dirty merge state against main, so the `pull_request`-triggered CI never starts (GitHub can't build the test-merge commit it runs against). Add a `push` trigger for the branch, which runs against the branch head directly and is unaffected by conflicts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumps the asset-canister recipe's sync plugin to the new `migration-v2.2.1-6b48585` release on [dfinity/certified-assets](https://github.com/dfinity/certified-assets/releases/tag/migration-v2.2.1-6b48585). ## Why A user reported that `icp deploy` failed during sync when the recipe's `dir` was a **nested** path such as `src/frontend/dist`: ``` failed to run plugin caused by: plugin returned error: canonicalize src/frontend/dist: No such file or directory (os error 44) ``` The v2.2.0 plugin called `canonicalize` during its scan step. Under WASI that calls `realpath`, which returns `ENOENT` for any path beneath a preopened directory whose guest name has more than one component (e.g. `src/frontend/dist`) — even though ordinary access works. Single-component dirs like `dist` happened to work, which is why our examples didn't catch it. The v2.2.1 plugin drops `canonicalize`; fix + regression test in dfinity/certified-assets#71. ## Change Only `recipes/asset-canister/recipe.hbs` — the plugin `url` + `sha256`: | | old (v2.2.0) | new (v2.2.1) | |---|---|---| | url | `…/migration-v2.2.0-209d688/sync_plugin.wasm` | `…/migration-v2.2.1-6b48585/sync_plugin.wasm` | | sha256 | `297c2ef0…` | `ca7cb566…` | The canister wasm is unchanged (still the legacy dfx `assetstorage` canister), so this is a drop-in patch bump for existing users. ## Releasing After merge, push the `asset-canister-v2.2.1` tag to trigger `release-recipe.yml` and publish the recipe release. Ref: https://forum.dfinity.org/t/icp-cli-announcements-and-feedback-discussion/60410/97
Ports the WASI `canonicalize` fix from the `migration` branch (#71) to `main`. ## Problem The sync plugin's scan step canonicalized the manifest `dir`. On `wasm32-wasip2`, `Path::canonicalize` calls `realpath`, which returns `ENOENT` for **any** path beneath a preopen whose WASI guest name has more than one component (e.g. `src/frontend/dist`) — even though `read_dir`/`metadata`/`read` through that preopen work fine. So a nested `dir` failed: ``` failed to run plugin caused by: plugin returned error: canonicalize src/frontend/dist: No such file or directory (os error 44) ``` Single-component dirs like `dist` happened to canonicalize to `/dist` and worked, which is why the existing fixtures never caught it. (Originally reported on the [forum](https://forum.dfinity.org/t/icp-cli-announcements-and-feedback-discussion/60410/97) against the migration recipe; same root cause lives here.) ## Fix Build the scan root by prepending `/` to the (host-validated, relative, `..`-free) manifest dir instead of canonicalizing. This yields the same shape canonicalize produced for single-component dirs (`/dist`, `/src/frontend/dist`) and uses only plain `wasi:filesystem` ops the preopen supports. `main`'s `scan()` had a single `canonicalize` call; `walk` already uses `read_dir` paths rooted at that dir, and `_headers`/`_redirects` load via relative paths — none of which need canonicalization. ## Test Adds an e2e fixture + `nested_dir_deploy` test using `dirs: [src/frontend/dist]` with a nested subdir asset. Verified it fails with the exact error above when the fix is reverted, and passes with it. - `cargo test -p sync-core` — 188 pass - `cargo test -p e2e --test sync` — 7 pass (incl. nested regression) - `cargo clippy -p sync-core` — clean Note: this is the same fix already shipped on `migration` as `migration-v2.2.1` (consumed by `asset-canister-v2.2.1`). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mainThe
migrationbranch is a permanently separate line of work. Do not merge it.Instead, we will git-tag commits on this branch and cut special GitHub releases that host the built
sync-pluginwasm module. icp-cli projects (recipes, templates) reference that hosted wasm directly.The driving use case: a companion assets recipe that swaps the sync mechanism from the old
type: assetstotype: pluginwhile keeping the canister wasm unchanged (the legacy dfx 0.32.0assetstoragecanister). This minimizes the upgrade effort for existing assets-recipe users when we droptype: assetssync in the upcoming icp-cli release.What this branch does
Retargets
sync-core/sync-pluginfrom this repo's V2 certified-assets canister to the legacyassetstoragecanister shipped with dfx 0.32.0 (ic-certified-assets, which reportsapi_version == 2). Per-asset metadata is read from.ic-assets.json5(fullic-assetparity) instead of the_headers/_redirectsfiles used onmain.The legacy canister's wire protocol is nearly identical to this repo's V2 canister; the differences are narrow and handled here:
main(new)get_redirect_rules/SetRedirectRulesCreateAssetArgumentsenable_aliasingSetAssetPropertiesArgumentsis_aliasedAssetPropertiesis_aliased_headers+_redirects.ic-assets.json5enable_aliasingsync-coreconfig.rs—.ic-assets.json5/.ic-assets.jsonparser ported fromic-asset: nested config inheritance, globmatch,ignore,headers,cache.max_age,enable_aliasing,allow_raw_access,encodings,security_policy.security_policy.rs—standard/hardened/disabledCSP presets.scan.rsrewritten asgather_asset_descriptors(walkdir + config-drivenignore;.well-knownkept; config files excluded).canister.rs— addenable_aliasing/is_aliased; dropRedirectRule/RulePattern/SetRedirectRules/get_redirect_rules.sync.rs— driveCreateAsset/SetAssetPropertiesfromAssetConfig; drop redirect & html-handling; keep chunk packing + commit staging.redirects.rs,headers.rs,html_handling.rs,glob.rs.Set-Cookie: ic_env=...header into every HTML asset (driven by icp-cli's env-var step) and returns it fromget_asset_properties.update_propertiesnow ignores that canister-managed cookie so repeat syncs report "up to date" instead of clobbering it every run.e2e
build.rsfetches the realassetstorage.wasm.gzfrom the dfx 0.32.0 release (sha256-pinned to the GH release digest, installed gzipped — no decompression) instead of building this repo's canister..ic-assets.json5; redirect / header / multi-dir cases removed; newconfig.rsend-to-end test (headers,max_age, aliasing,security_policyCSP, config-file-not-uploaded).Test plan
cargo test -p sync-core— 75 unit tests pass.cargo build -p sync-plugin --target wasm32-wasip2 --release— links under WASI.cargo test -p e2e— deploys the real 0.32.0assetstorage.wasmand exercises the plugin viaicp deploy; config test + all sync tests (incl. idempotentno_op_sync) pass.🤖 Generated with Claude Code