diff --git a/docs/superpowers/plans/2026-09-18-wasm-registry-v2.md b/docs/superpowers/plans/2026-09-18-wasm-registry-v2.md new file mode 100644 index 0000000..fad502c --- /dev/null +++ b/docs/superpowers/plans/2026-09-18-wasm-registry-v2.md @@ -0,0 +1,333 @@ +# WASM Registry Contract v2 + hello v2 (sub-project B) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** The plugin registry lists WebAssembly plugins only — manifest `runtime: wasm`, the `.wasm` shipped as a release asset, declared `allowed_hosts` — and `goblogplatform/goblog-plugin-hello` v2.0.0 is the first such plugin. + +**Architecture:** `goblog-plugin-hello` is rewritten as a standard-Go WASM plugin with a release workflow that builds and uploads `plugin.wasm`. In the registry, `Release` carries its assets, `Source` gains `ReleaseAsset`, `ValidateEntry` downloads the asset named by the manifest's `entry` and validates it with `goblog validate-plugin plugin.wasm` in the pinned `compscidr/goblog:v0.2.9` image; `Build` emits `runtime`, `allowed_hosts`, `install_type: wasm` and the asset's browser download URL. + +**Tech Stack:** Go (go-github v92: `RepositoryRelease.Assets`, `DownloadReleaseAsset`), `github.com/extism/go-pdk` v1.1.3 in the hello repo, GitHub Actions. + +**Spec:** `docs/superpowers/specs/2026-09-18-wasm-plugins-design.md` §3. + +## Global Constraints + +- Image pin everywhere: `compscidr/goblog:v0.2.9` (workflows, `cmd/registry/main.go` default, `docs/CONTRACT.md`, `validator_test.go`). +- Manifest v2: `runtime` required and must be `"wasm"`; `entry` defaults to `plugin.wasm`, must end in `.wasm`, no `/`; `allowed_hosts` optional list, each `^[A-Za-z0-9.*:-]+$` (hostnames / IPs / globs, no scheme or path); other fields unchanged. A manifest without `runtime: wasm` fails with "the directory only lists WebAssembly plugins; see docs/CONTRACT.md". +- The entry is a **release asset** of the validated release (name == `entry`); missing → `release vX.Y.Z has no asset named plugin.wasm`; asset ≤ 16 MiB. +- Index entry: `download_url` = the asset's `browser_download_url`; `sha256` of the asset bytes; `install_type: "wasm"`, `runtime: "wasm"`, `allowed_hosts: [...]` (empty array, never null). +- `validate-plugin` output must report `"runtime":"wasm"`; name/version identity checks unchanged. +- hello v2: identity `hello` / `Hello` / `2.0.0`, settings `enabled` (default `true`) and `message` (default `Hello from a WebAssembly plugin`), `template_footer` renders `

` + HTML-escaped message when enabled; built with `GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -ldflags="-s -w" -o plugin.wasm .`; release workflow uploads the asset on `release: published`. +- Repos: `~/dev/goblog-plugin-hello` (branch `feat/wasm`, PR, then release `v2.0.0` after merge), `~/dev/goblog-plugins` (branch `feat/wasm-contract`, one PR). Never push to `main`; never merge; commit trailer `Co-Authored-By: Claude Opus 5 (1M context) `; `git branch --show-current` before every commit. +- Toolchain quirks on this machine: prefix `go` with `GOTOOLCHAIN=go1.26.1` when it complains; `TMPDIR=$HOME/.cache/goblog-registry` for anything that runs Docker. + +--- + +### Task 1: hello v2.0.0 as a WASM plugin + +**Files (in `~/dev/goblog-plugin-hello`):** +- Create: `go.mod`, `main.go`, `.github/workflows/release.yml`, `.gitignore` +- Delete: `plugin.go` +- Modify: `goblog-plugin.json`, `README.md`, `CHANGELOG.md` + +- [ ] **Step 1: Rewrite** + +`go.mod`: +``` +module github.com/goblogplatform/goblog-plugin-hello + +go 1.25.0 + +toolchain go1.26.1 + +require github.com/extism/go-pdk v1.1.3 +``` + +`main.go`: +```go +// Hello is the reference goblog WebAssembly plugin: it appends a +// configurable greeting to the footer of every page. +// +// Build: GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -ldflags="-s -w" -o plugin.wasm . +// Every export takes JSON on stdin (pdk.Input) and returns JSON or HTML +// (pdk.Output). See goblog's README "WebAssembly plugins" for the contract. +package main + +import ( + "encoding/json" + "html" + + pdk "github.com/extism/go-pdk" +) + +// hookInput is the ctx goblog passes to template hooks; only settings are +// needed here. +type hookInput struct { + Settings map[string]string `json:"settings"` +} + +//go:wasmexport identity +func identity() int32 { + pdk.OutputString(`{"name":"hello","display_name":"Hello","version":"2.0.0"}`) + return 0 +} + +//go:wasmexport settings +func settings() int32 { + pdk.OutputString(`[` + + `{"key":"enabled","type":"text","default":"true","label":"Enabled","description":"Set to 'true' to show the greeting"},` + + `{"key":"message","type":"text","default":"Hello from a WebAssembly plugin","label":"Message","description":"Text shown at the bottom of every page"}` + + `]`) + return 0 +} + +//go:wasmexport template_footer +func templateFooter() int32 { + var in hookInput + if err := json.Unmarshal(pdk.Input(), &in); err != nil { + pdk.SetErrorString("template_footer: " + err.Error()) + return 1 + } + if in.Settings["enabled"] != "true" { + return 0 + } + // Always escape setting values: an admin typed them, but the browser + // will trust whatever this returns. + pdk.OutputString(`

` + html.EscapeString(in.Settings["message"]) + `

`) + return 0 +} + +func main() {} +``` + +`.gitignore`: `plugin.wasm` + +`goblog-plugin.json`: +```json +{ + "name": "hello", + "display_name": "Hello", + "description": "Appends a configurable greeting to the footer of every page. The reference goblog WebAssembly plugin.", + "author": "Jason Ernst", + "license": "Apache-2.0", + "runtime": "wasm", + "entry": "plugin.wasm", + "allowed_hosts": [], + "min_goblog_version": "0.2.9", + "homepage": "https://github.com/goblogplatform/goblog-plugin-hello" +} +``` + +`.github/workflows/release.yml`: +```yaml +name: Release +on: + release: + types: [published] +permissions: + contents: write +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-go@v7 + with: + go-version-file: go.mod + - name: Build plugin.wasm + run: GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -ldflags="-s -w" -o plugin.wasm . + - name: Upload to the release + env: + GH_TOKEN: ${{ github.token }} + run: gh release upload "${{ github.event.release.tag_name }}" plugin.wasm --clobber +``` + +`README.md`: +````markdown +# goblog-plugin-hello + +The reference [goblog](https://github.com/goblogplatform/goblog) WebAssembly plugin. It appends a configurable greeting to the footer of every page — the smallest thing that proves a plugin is loaded. + +## Install + +From your goblog's **Admin → Plugins → Browse**, search for *Hello* and click **Install** (goblog 0.2.9 or newer). Or download `plugin.wasm` from the [latest release](https://github.com/goblogplatform/goblog-plugin-hello/releases/latest) into `plugins/wasm/` and restart goblog. + +## Settings + +Under **Admin → Settings → Hello**: + +| Setting | Default | Meaning | +|---|---|---| +| `enabled` | `true` | Set to `false` to hide the greeting | +| `message` | `Hello from a WebAssembly plugin` | The text shown at the bottom of every page | + +## Build it yourself + +```bash +GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -ldflags="-s -w" -o plugin.wasm . +``` + +Needs Go 1.24 or newer. The plugin talks to no network (`allowed_hosts` is empty), stores nothing, and only implements the `identity`, `settings` and `template_footer` exports. + +## Use it as a template + +Copy this repository, change the identity (`name` must be unique — it keys the plugin's settings) and follow the [plugin directory contract](https://github.com/goblogplatform/plugins/blob/main/docs/CONTRACT.md). Tag a release as `vX.Y.Z`; the workflow builds and uploads `plugin.wasm` for you. + +## License + +Apache-2.0. +```` + +`CHANGELOG.md` — prepend: +```markdown +## 2.0.0 + +- Rewritten as a WebAssembly plugin (sandboxed; installable from the directory in goblog ≥ 0.2.9). The 1.x Yaegi `.go` file is no longer shipped. +``` + +- [ ] **Step 2: Build and validate locally** + +```bash +cd ~/dev/goblog-plugin-hello && git checkout -b feat/wasm main +GOTOOLCHAIN=go1.26.1 go mod tidy +GOOS=wasip1 GOARCH=wasm GOTOOLCHAIN=go1.26.1 go build -buildmode=c-shared -ldflags="-s -w" -o plugin.wasm . && ls -la plugin.wasm +mkdir -p $HOME/.cache/goblog-registry/hello && cp plugin.wasm $HOME/.cache/goblog-registry/hello/ +docker run --rm --network none -v "$HOME/.cache/goblog-registry/hello:/p:ro" --entrypoint /go/src/github.com/compscidr/goblog/goblog compscidr/goblog:v0.2.9 validate-plugin /p/plugin.wasm +``` +Expected: `{"name":"hello","display_name":"Hello","version":"2.0.0","runtime":"wasm"}`. (If the `v0.2.9` image isn't on Docker Hub yet, use goblog's local build: `cd ~/dev/goblog && go run . validate-plugin $HOME/.cache/goblog-registry/hello/plugin.wasm`.) + +Also drop it into a local goblog to see the footer: copy to `~/dev/goblog/plugins/wasm/hello.wasm`, run goblog, `curl localhost:7000/ | grep 'Hello from a WebAssembly plugin'`, then remove the file. + +- [ ] **Step 3: Commit, PR, then (after the maintainer merges) release** + +```bash +git add -A && git commit -m "Rewrite as a WebAssembly plugin (v2.0.0) + +Co-Authored-By: Claude Opus 5 (1M context) " +git push -u origin feat/wasm +gh pr create --base main --title "Rewrite as a WebAssembly plugin (v2.0.0)" --body "..." +``` +After merge: `gh release create v2.0.0 --title v2.0.0 --notes "Rewritten as a WebAssembly plugin; install from Admin → Plugins (goblog ≥ 0.2.9)."`, then `gh run watch` the `Release` workflow and `gh release view v2.0.0 --json assets --jq '.assets[].name'` → `plugin.wasm`. + +--- + +### Task 2: Registry — manifest v2, release assets, wasm validation, index fields + +**Files (in `~/dev/goblog-plugins`, branch `feat/wasm-contract`):** +- Modify: `internal/registry/manifest.go` (+test), `source.go` (+test, `fakeGitHub`), `validate.go` (+test, `memSource`), `validator.go` (+test), `build.go` (+test), `cmd/registry/main_test.go` (`memSource`), `cmd/registry/main.go` (image default), `.github/workflows/{validate,publish,submit}.yml` (image), `docs/CONTRACT.md`, `README.md` + +**Interfaces:** +- `Manifest` gains `Runtime string \`json:"runtime"\``, `AllowedHosts []string \`json:"allowed_hosts"\``. +- `type Asset struct{ ID int64; Name string; Size int; DownloadURL string }`; `Release.Assets []Asset`. +- `Source.ReleaseAsset(ctx, owner, repo string, assetID int64) ([]byte, error)` (≤ 16 MiB). +- `Info` gains `Runtime string \`json:"runtime"\``; `DockerValidator` writes and validates `plugin.wasm`. +- `Validated.Entry` = asset bytes; `Validated.Asset Asset`. +- `IndexEntry` gains `Runtime`, `AllowedHosts []string` (json `runtime`, `allowed_hosts`). + +- [ ] **Step 1: Failing tests** + +`manifest_test.go`: update `goodManifest` to include `"runtime": "wasm", "entry": "plugin.wasm", "allowed_hosts": ["api.example.test"]`; `TestParseManifest_Good` asserts `Runtime == "wasm"`, `Entry == "plugin.wasm"`, `AllowedHosts` length 1; `TestParseManifest_EntryDefaultsToPluginGo` → renamed `..._EntryDefaultsToPluginWasm` expecting `plugin.wasm`; error cases add `"missing runtime"` (remove the field → error containing "WebAssembly"), `"go runtime"` (`"runtime": "go"`), `"entry not wasm"` (`plugin.go`), `"bad host"` (`"allowed_hosts": ["https://x"]`). + +`source_test.go`: releases fixture gets `"assets":[{"id":11,"name":"plugin.wasm","size":3,"browser_download_url":"https://github.com/o/r/releases/download/v1.1.0/plugin.wasm"}]` on v1.1.0; handler `GET /repos/o/r/releases/assets/11` writes bytes `wasm` with `Content-Type: application/octet-stream`; test asserts `rels[0].Assets[0].Name == "plugin.wasm"`, `DownloadURL` set, and `src.ReleaseAsset(ctx,"o","r",11)` returns `"wasm"`; unknown id → error. + +`validate_test.go` `memSource`: add `assets map[int64][]byte` and `ReleaseAsset`; `helloSource()`'s v1.1.0 release gets `Assets: []Release{...}` → `[]Asset{{ID: 11, Name: "plugin.wasm", Size: len(helloWasm), DownloadURL: "https://github.com/o/hello/releases/download/v1.1.0/plugin.wasm"}}` with `helloWasm = []byte("\x00asm hello v1.1.0")`; `helloValidator()` keyed on `sum(helloWasm)` returning `Info{Name:"hello", DisplayName:"Hello", Version:"1.1.0", Runtime:"wasm"}`; drop the `plugin.go` file entry. New error cases: `"no asset"` (remove Assets → error containing "no asset named plugin.wasm"), `"asset too big"` (Size > 16 MiB → error containing "16"), `"not wasm runtime"` (validator returns `Runtime: ""` → error containing "runtime"). `TestValidateEntry_Good` asserts `v.Asset.DownloadURL` and `v.SHA256 == sum(helloWasm)`. + +`validator_test.go`: `TestDockerValidator_CommandShape` expects `validate-plugin /p/plugin.wasm`; `TestDockerValidator_Real` uses the real `echo.wasm` from goblog (`../../../goblog/plugin/wasm/testdata/echo.wasm` relative to the package — read it; skip if absent) with image `compscidr/goblog:v0.2.9` and asserts `Runtime == "wasm"`. + +`build_test.go`: `want` entry gets `DownloadURL: "https://github.com/o/hello/releases/download/v1.1.0/plugin.wasm"`, `InstallType: "wasm"`, `Runtime: "wasm"`, `AllowedHosts: []string{"api.example.test"}`, `SHA256: sum(helloWasm)`; a second entry without hosts must serialise `"allowed_hosts": []` (assert the raw JSON contains `"allowed_hosts": []`). + +`cmd/registry/main_test.go`: fixture manifest gains `"runtime":"wasm"`; release gets an asset; `memSource.ReleaseAsset` returns fixed bytes; `okValidator` returns `Runtime: "wasm"`. + +Run: `GOTOOLCHAIN=go1.26.1 go test ./... 2>&1 | head` → FAIL to compile. + +- [ ] **Step 2: Implement** + +`manifest.go`: +```go + Runtime string `json:"runtime"` + AllowedHosts []string `json:"allowed_hosts"` +``` +`ParseManifest`: default `Entry = "plugin.wasm"`; checks: `if m.Runtime != "wasm" { problems = append(problems, "runtime must be \"wasm\": the directory only lists WebAssembly plugins; see docs/CONTRACT.md") }`; entry `^[A-Za-z0-9_.-]+\.wasm$` ("entry must be a .wasm release asset name"); each host `^[A-Za-z0-9.*:-]+$` and non-empty ("allowed_hosts entries must be hostnames, IPs or globs without scheme or path"); `if m.AllowedHosts == nil { m.AllowedHosts = []string{} }`. + +`source.go`: +```go +// Asset is a file attached to a GitHub release. +type Asset struct { + ID int64 + Name string + Size int + DownloadURL string // browser_download_url +} +``` +`Release.Assets []Asset`; in `Releases()` map `r.Assets` (`a.GetID()`, `a.GetName()`, `a.GetSize()`, `a.GetBrowserDownloadURL()`). Interface + impl: +```go + // ReleaseAsset downloads a release asset by id (at most MaxAssetBytes). + ReleaseAsset(ctx context.Context, owner, repo string, assetID int64) ([]byte, error) +``` +```go +const MaxAssetBytes = 16 << 20 + +func (g *GitHubSource) ReleaseAsset(ctx context.Context, owner, repo string, assetID int64) ([]byte, error) { + rc, _, err := g.client.Repositories.DownloadReleaseAsset(ctx, owner, repo, assetID, http.DefaultClient) + if err != nil { + return nil, fmt.Errorf("download asset %d of %s/%s: %w", assetID, owner, repo, err) + } + defer rc.Close() + b, err := io.ReadAll(io.LimitReader(rc, MaxAssetBytes+1)) + if err != nil { + return nil, fmt.Errorf("download asset %d of %s/%s: %w", assetID, owner, repo, err) + } + if len(b) > MaxAssetBytes { + return nil, fmt.Errorf("asset %d of %s/%s exceeds %d bytes", assetID, owner, repo, MaxAssetBytes) + } + return b, nil +} +``` +(In the httptest fake, `DownloadReleaseAsset` first GETs the API path with `Accept: application/octet-stream`; serve the bytes there with status 200 and no redirect.) + +`validate.go`: after the manifest + README checks, find the asset: +```go + var asset *Asset + for i := range latest.Assets { + if latest.Assets[i].Name == manifest.Entry { + asset = &latest.Assets[i] + } + } + if asset == nil { + return nil, fmt.Errorf("%s: release %s has no asset named %s (the release workflow must upload it)", repo, latest.Tag, manifest.Entry) + } + if asset.Size > MaxAssetBytes { + return nil, fmt.Errorf("%s@%s: asset %s is %d bytes; the limit is %d (16 MiB)", repo, latest.Tag, asset.Name, asset.Size, MaxAssetBytes) + } + entry, err := src.ReleaseAsset(ctx, owner, name, asset.ID) + ... + info, err := val.Validate(ctx, entry) + ... + if info.Runtime != "wasm" { + return nil, fmt.Errorf("%s@%s: %s is not a WebAssembly plugin (runtime %q)", repo, latest.Tag, manifest.Entry, info.Runtime) + } +``` +`Validated` gains `Asset Asset`; the `File` fetch of `manifest.Entry` is removed. + +`validator.go`: `Info.Runtime string \`json:"runtime"\``; write `plugin.wasm`; args end `"validate-plugin", "/p/plugin.wasm"`; comment updated. + +`build.go`: `IndexEntry` gains `Runtime string \`json:"runtime"\``, `AllowedHosts []string \`json:"allowed_hosts"\``; `buildDetail` sets `DownloadURL: v.Asset.DownloadURL`, `InstallType: "wasm"`, `Runtime: "wasm"`, `AllowedHosts: v.Manifest.AllowedHosts` (never nil). + +Image pin → `compscidr/goblog:v0.2.9` in `cmd/registry/main.go`, the three workflows, `docs/CONTRACT.md`, `validator_test.go`. + +- [ ] **Step 3: Docs** + +`docs/CONTRACT.md` — rewrite for wasm: repository must contain `goblog-plugin.json` (fields incl. `runtime: "wasm"`, `entry`, `allowed_hosts`), `README.md`, source, `LICENSE`; releases `vX.Y.Z` **with `plugin.wasm` attached** (show the copy-paste `release.yml` from Task 1 and the build command); host contract summary with a link to goblog's README section; `allowed_hosts` semantics (exact host or glob; empty = no network; shown to users as "Talks to"); local check `docker run … validate-plugin /p/plugin.wasm` (expect `"runtime":"wasm"`); submission via the issue form unchanged. Say plainly: `.go` (Yaegi) plugins are not accepted in the directory. +`README.md` — update the index description (`runtime`, `allowed_hosts`, `download_url` is the release asset). + +- [ ] **Step 4: Verify, PR** + +`gofmt -l . && GOTOOLCHAIN=go1.26.1 go vet ./... && GOTOOLCHAIN=go1.26.1 go test ./...` → PASS. Real run (after hello v2.0.0 is released with its asset): `TMPDIR=$HOME/.cache/goblog-registry GITHUB_TOKEN=$(gh auth token) GOTOOLCHAIN=go1.26.1 go run ./cmd/registry build --out /tmp/dist-v2` → `hello: built`; `dist/index.json` shows `"runtime": "wasm"`, `"install_type": "wasm"`, `download_url` ending `/releases/download/v2.0.0/plugin.wasm`, `sha256` equal to `sha256sum` of the downloaded asset. + +Commit(s), push `feat/wasm-contract`, `gh pr create` ("Registry contract v2: WebAssembly plugins with release assets"), `gh pr checks --watch` → `Validate` passes against live hello v2.0.0. + +--- + +### Task 3: Post-merge verification + +- [ ] After the maintainer merges the registry PR: watch `Publish index`, then `curl -fsS https://goblogplatform.github.io/plugins/index.json` → one entry `hello 2.0.0` with `runtime: wasm`; download the asset from `download_url` and compare `sha256sum` with the index; `curl https://www.goblog.live/plugins` shows `wasm` and "Talks to: no network"/empty (whatever the listing renders for no hosts). Install from a goblog running v0.2.9 (local is fine: `go run .` with `plugin_directory_url` default) via `POST /api/v1/plugins/install {"name":"hello"}` → 200 and the footer greeting appears. diff --git a/docs/superpowers/plans/2026-09-19-goblog-v0.3.0-scholar-removal.md b/docs/superpowers/plans/2026-09-19-goblog-v0.3.0-scholar-removal.md new file mode 100644 index 0000000..005ad42 --- /dev/null +++ b/docs/superpowers/plans/2026-09-19-goblog-v0.3.0-scholar-removal.md @@ -0,0 +1,149 @@ +# goblog v0.3.0 — remove the compiled-in scholar plugin (sub-project D) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** goblog no longer ships the scholar plugin (it lives in the directory as `goblogplatform/goblog-plugin-scholar`), pages whose plugin isn't installed are hidden instead of rendering empty, and goblog.live / jasonernst.com are deployed with a persisted `plugins/wasm/` so the WASM scholar can be installed. + +**Architecture:** Delete `plugins/scholar` and the `compscidr/scholar` dependency. Treat any page type that is neither built-in (`writing`, `about`, `custom`, `tags`, `archives`) nor owned by a registered plugin as *unowned*: hidden from nav/sitemap and answered with the existing "Page Not Available" 404 — so a `research` row survives the upgrade and comes back the moment the WASM scholar is installed. iac adds the bind mounts. + +**Spec:** `docs/superpowers/specs/2026-09-18-wasm-plugins-design.md` §5. + +## Global Constraints + +- Breaking change → **v0.3.0**; release notes must say: Scholar Publications is now a directory plugin — install it from Admin → Plugins; `enabled`, `semantic_scholar_id`, `semantic_scholar_api_key`, `article_limit` carry over; Google Scholar scraping is gone (`source`, `scholar_id` are ignored) — set a Semantic Scholar author id. +- Built-in page types: `blog.PageTypeWriting`, `PageTypeAbout`, `PageTypeCustom`, `PageTypeTags`, `PageTypeArchives`. Anything else without an owning plugin is unowned. +- `tools/migrate.go`: the fresh-install seed no longer creates a `research` page (the plugin's `pages` declaration creates it on install); existing rows are untouched. +- goblog: branch `feat/v0.3.0-scholar-removal`; iac: branch `feat/wasm-plugin-mounts`. Never push to `main`; never merge; commit trailer `Co-Authored-By: Claude Opus 5 (1M context) `; `git branch --show-current` before every commit. + +--- + +### Task 1: goblog — remove the plugin, hide unowned pages, docs + +**Files:** +- Delete: `plugins/scholar/` (both files) +- Modify: `goblog.go` (import + `registry.Register(scholarplugin.New())`; `PageFilter`), `go.mod`/`go.sum` (`go mod tidy`), `blog/blog.go` (`DynamicPage` default branch), `blog/blog_test.go`, `tools/migrate.go` (seed), `plugins/directory/directory.go` (comment), `plugins/dynamic/hello.go.example` (header), `README.md` +- Test: `blog/blog_test.go`, `admin` tests if any reference scholar (`grep -rn scholar --include=*_test.go`) + +- [ ] **Step 1: Failing test** — append to `blog/blog_test.go`: + +```go +// TestUnownedPluginPageIsHidden: a page whose plugin type has no registered +// owner (e.g. "research" after the scholar plugin moved to the directory) is +// hidden from the nav and answers 404 until a plugin claims it again. +func TestUnownedPluginPageIsHidden(t *testing.T) { + db, _ := gorm.Open(sqlite.Open(":memory:")) + db.AutoMigrate(&auth.BlogUser{}, &blog.PostType{}, &blog.Post{}, &blog.Tag{}, &blog.Comment{}, &blog.Page{}, &blog.Setting{}, &plugin.PluginSetting{}) + db.Create(&blog.Page{Title: "Research", Slug: "research", PageType: "research", ShowInNav: true, Enabled: true}) + db.Create(&blog.Page{Title: "About", Slug: "about", PageType: blog.PageTypeAbout, ShowInNav: true, Enabled: true, Content: "about"}) + a := &Auth{} + a.On("IsAdmin", mock.Anything).Return(false) + a.On("IsLoggedIn", mock.Anything).Return(false) + b := blog.New(db, a, "test") + reg := plugin.NewRegistry(db) // no plugin owns "research" + b.PageFilter = blog.PluginPageFilter(reg) + + router := gin.New() + router.Use(plugin.Middleware(reg)) + tmpl := template.Must(template.New("").Funcs(template.FuncMap{ + "rawHTML": func(s string) template.HTML { return template.HTML(s) }, + }).ParseGlob("../templates/shared/*.html")) + template.Must(tmpl.ParseGlob("../themes/default/templates/*.html")) + router.SetHTMLTemplate(tmpl) + router.NoRoute(b.NoRoute) + + get := func(path string) *httptest.ResponseRecorder { + w := httptest.NewRecorder() + req, _ := http.NewRequest("GET", path, nil) + router.ServeHTTP(w, req) + return w + } + if w := get("/research"); w.Code != http.StatusNotFound || !strings.Contains(w.Body.String(), "Page Not Available") { + t.Errorf("/research without an owner: code=%d", w.Code) + } + if w := get("/about"); w.Code != http.StatusOK { + t.Errorf("/about: code=%d", w.Code) + } + for _, p := range b.GetNavPages() { + if p.PageType == "research" { + t.Error("unowned research page must not be in the nav") + } + } + // A plugin claiming the type brings the page back. + reg.Register(&subPathPlugin{}) // owns "dir" + db.Create(&blog.Page{Title: "Dir", Slug: "dir", PageType: "dir", ShowInNav: true, Enabled: true}) + reg.Init() + found := false + for _, p := range b.GetNavPages() { + found = found || p.PageType == "dir" + } + if !found { + t.Error("an owned plugin page must be in the nav") + } +} +``` +(`subPathPlugin` already exists in this test file from the sub-path work; its `Settings()` declares `enabled` default `true`.) + +- [ ] **Step 2: Implement** + +`blog/blog.go` — add: +```go +// builtinPageTypes are rendered by blog itself; any other page type needs a +// registered plugin to own it. +var builtinPageTypes = map[string]bool{PageTypeWriting: true, PageTypeAbout: true, PageTypeCustom: true, PageTypeTags: true, PageTypeArchives: true} + +// PluginPageFilter hides pages whose type belongs to a disabled plugin or to +// no plugin at all (e.g. a plugin that was uninstalled or moved to the +// directory); such pages come back as soon as a plugin claims the type. +func PluginPageFilter(reg interface { + HasPageType(string) bool + IsPageTypeEnabled(string) bool +}) PageFilter { + return func(page Page) bool { + if builtinPageTypes[page.PageType] { + return true + } + if !reg.HasPageType(page.PageType) { + return false + } + return reg.IsPageTypeEnabled(page.PageType) + } +} +``` +In `DynamicPage`'s `default:` branch, after the plugin-handled block: if `!builtinPageTypes[page.PageType]` and (`r == nil || !r.HasPageType(page.PageType)`), render the existing "Page Not Available / This page is currently disabled." 404 (reuse the same `gin.H`) with description "This page's plugin is not installed." and return — instead of falling through to the custom-content fallback. Keep the fallback for `custom`. + +`goblog.go`: replace the inline `PageFilter` closure with `_blog.PageFilter = blog.PluginPageFilter(registry)`; delete the scholar import and registration. `go mod tidy` removes `compscidr/scholar`. + +`tools/migrate.go`: delete the `Research` entry from the seeded pages (and the `ScholarID` field use); leave `knownPrefixes`. + +`plugins/directory/directory.go`: fix the comment that references the scholar plugin creating its page. `plugins/dynamic/hello.go.example`: the header line about the hello repo now says "…is published as a WebAssembly plugin at github.com/goblogplatform/goblog-plugin-hello". + +`README.md`: Features — replace the Research/scholar bullet with "Research page listing your publications from Semantic Scholar — install **Scholar Publications** from Admin → Plugins"; built-in plugins list drops `scholar`; the hook table's example plugin becomes `plugins/directory`; add an "Upgrading to 0.3.0" note under Plugins: what to do (install Scholar Publications; settings carry over; set `semantic_scholar_id` if you used Google Scholar). + +- [ ] **Step 3: Verify, commit, PR** + +`go build -v . && go vet ./... && go test -race goblog/... -count=1` → PASS (installer tests that used a compiled-in `scholar` entry keep working — they register their own `compiledPlugin`). Manual: `go run .` with an existing DB that has a `research` row → nav lacks Research, `/research` → 404 "not installed"; drop `~/dev/goblog-plugin-scholar/plugin.wasm` + sidecar into `plugins/wasm/`, restart → Research is back and renders. + +```bash +git checkout -b feat/v0.3.0-scholar-removal main +git add -A && git commit -m "Remove the compiled-in scholar plugin; hide pages whose plugin is not installed + +Co-Authored-By: Claude Opus 5 (1M context) " +git push -u origin feat/v0.3.0-scholar-removal +gh pr create --base main --title "v0.3.0: Scholar Publications moves to the plugin directory" --body "..." +``` + +--- + +### Task 2: iac — persisted `plugins/wasm` on goblog.live and jasonernst.com + +**Files (in `~/dev/iac`, branch `feat/wasm-plugin-mounts`):** `ansible/roles/projects/tasks/main.yml`, `ansible/roles/jasonernst_com/tasks/main.yml`. + +- [ ] goblog.live: add `plugins-wasm` to the directories loop and the volume `- /opt/goblog-live/plugins-wasm:/go/src/github.com/compscidr/goblog/plugins/wasm`. jasonernst.com prod + staging: the same for `/opt/goblog/prod/plugins-wasm` and `/opt/goblog/staging/plugins-wasm` (mirror how `uploads` is done in that role — read it first). Don't bump image tags here (Renovate/v0.3.0 bump is separate). Validate with `ansible-lint`/YAML load; commit; push; PR "Persist plugins/wasm for goblog sites" noting it's needed for one-click plugin installs to survive redeploys. + +--- + +### Task 3: Release and go-live (after merges) + +- [ ] After the goblog PR merges: `gh release create v0.3.0 --generate-notes` with the breaking-change paragraph prepended; wait for the image. +- [ ] Scholar plugin: bump `min_goblog_version` to `0.3.0` in `goblog-plugin-scholar` (PR, merge), then release `v2.0.0`; then PR to `goblogplatform/plugins` adding ` - repo: goblogplatform/goblog-plugin-scholar` to `registry.yaml` (`Validate` runs the real check). +- [ ] iac: merge the mounts PR and the v0.3.0 bump(s); run `projects.yml --tags goblog-live` and `jasonernst_com.yml --limit www --tags website` (Jason; 1Password); on each site install Scholar Publications from Admin → Plugins, set `semantic_scholar_id` where it was Google Scholar, enable; verify `/research`. diff --git a/docs/superpowers/plans/2026-09-19-scholar-wasm-plugin.md b/docs/superpowers/plans/2026-09-19-scholar-wasm-plugin.md new file mode 100644 index 0000000..3fcc9af --- /dev/null +++ b/docs/superpowers/plans/2026-09-19-scholar-wasm-plugin.md @@ -0,0 +1,681 @@ +# Scholar WASM Plugin (sub-project C) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** `goblogplatform/goblog-plugin-scholar` — the Research/publications page as a directory-installable WebAssembly plugin, replacing goblog's compiled-in scholar plugin. + +**Architecture:** Standard Go compiled to wasip1. Pure logic (Semantic Scholar JSON → articles, sorting, HTML rendering, cache freshness) lives in files with no PDK imports and is unit-tested natively; the exports and host calls (`pdk` HTTP, `store_get/set`) live in `//go:build wasip1` files. `render_page` serves from the store cache, fetching when missing/stale; a `refresh` job keeps it warm. + +**Tech Stack:** Go 1.25 (toolchain 1.26), `github.com/extism/go-pdk` v1.1.3, GitHub Actions release workflow (as in hello v2). + +**Spec:** `docs/superpowers/specs/2026-09-18-wasm-plugins-design.md` §4. + +## Global Constraints + +- Identity `scholar` / `Scholar Publications` / `2.0.0`; page type `research`, slug `research`, title `Research`, `show_in_nav` true, `nav_order` 20 — same as the compiled-in plugin so settings and the page row carry over. +- Settings (keys must match the old ones where they exist): `enabled` (default `false`), `semantic_scholar_id`, `semantic_scholar_api_key`, `article_limit` (default `50`), `cache_hours` (default `24`). Google Scholar is dropped. +- Store key `articles` holds `{"fetched_at": RFC3339, "articles": [...]}`; a render with a fresh cache makes no HTTP call; stale/missing → fetch, store, render; fetch failure with a cache → render the stale cache; with no cache → the notice ``; missing author id → ``. +- Semantic Scholar: `GET https://api.semanticscholar.org/graph/v1/author/{id}/papers?fields=title,authors,year,publicationDate,venue,journal,citationCount,url&limit=100&offset=N`, paginate via `next` until `article_limit` papers; header `x-api-key` when set; `allowed_hosts: ["api.semanticscholar.org"]`. +- HTML matches the compiled-in `renderArticlesHTML`: per article a bordered div with title (linked only through `safeHref` — http/https only), authors line, meta line `year · journal · N citations`; empty → `

No publications found.

`. Sort: year desc, then publication date desc, then citations desc. +- Jobs: `refresh` every 3600 s; `run_job` re-fetches only when the cache is older than `cache_hours` (so the job interval doesn't need to match the setting), and only when `enabled` and an id is set. +- Repo `goblogplatform/goblog-plugin-scholar` (public, Apache-2.0), checkout `~/dev/goblog-plugin-scholar`; branch `feat/initial`, PR, release `v2.0.0` after merge (the maintainer merges; the controller releases). Commit trailer `Co-Authored-By: Claude Opus 5 (1M context) `. + +--- + +## File structure + +| File | Build | Responsibility | +|---|---|---| +| `go.mod` | | module `github.com/goblogplatform/goblog-plugin-scholar`, go 1.25.0, toolchain go1.26.1, go-pdk v1.1.3 | +| `article.go` | all | `Article`, `parsePapersPage`, `sortArticles`, `renderArticlesHTML`, `safeHref`, cache freshness | +| `article_test.go` | native | unit tests with a recorded API page | +| `fetch.go` | all | `fetchAll(get getter, id, key string, limit int) ([]Article, error)` — pagination over an injected `getter` | +| `main.go` | wasip1 | exports, host imports, PDK HTTP `getter`, store cache | +| `main_native.go` | !wasip1 | `func main() {}` so native `go test` compiles | +| `goblog-plugin.json`, `README.md`, `CHANGELOG.md`, `LICENSE`, `.gitignore`, `.github/workflows/release.yml` | | packaging | + +--- + +### Task 1: Repo, pure logic with native tests + +- [ ] **Step 1: Create the repo** + +```bash +cd ~/dev && gh repo create goblogplatform/goblog-plugin-scholar --public \ + --description "goblog plugin: a Research page listing your publications from Semantic Scholar (WebAssembly)" --clone +cd ~/dev/goblog-plugin-scholar && git checkout -b feat/initial 2>/dev/null || git checkout -b feat/initial +cp ~/dev/goblog/LICENSE LICENSE; printf 'plugin.wasm\n*.test\n' > .gitignore +cat > go.mod <<'EOM' +module github.com/goblogplatform/goblog-plugin-scholar + +go 1.25.0 + +toolchain go1.26.1 + +require github.com/extism/go-pdk v1.1.3 +EOM +``` + +- [ ] **Step 2: Failing tests** — `article_test.go`: + +```go +package main + +import ( + "strings" + "testing" + "time" +) + +const pageJSON = `{"offset":0,"next":2,"data":[ + {"paperId":"p1","url":"https://www.semanticscholar.org/paper/p1","title":"Old & Cited","year":2019,"publicationDate":"2019-03-01","venue":"Conf A","journal":{"name":"J. A"},"citationCount":40,"authors":[{"name":"A. One"},{"name":"B. Two"}]}, + {"paperId":"p2","url":"javascript:alert(1)","title":"","year":2024,"publicationDate":"2024-06-01","venue":"","journal":null,"citationCount":1,"authors":[{"name":"C. Three"}]} +]}` + +func TestParsePapersPage(t *testing.T) { + page, err := parsePapersPage([]byte(pageJSON)) + if err != nil { + t.Fatal(err) + } + if page.Next == nil || *page.Next != 2 || len(page.Articles) != 2 { + t.Fatalf("page = %+v", page) + } + a := page.Articles[0] + if a.Title != "Old & Cited" || a.Authors != "A. One, B. Two" || a.Year != 2019 || a.Journal != "J. A" || a.Citations != 40 || a.URL != "https://www.semanticscholar.org/paper/p1" || a.Date != "2019-03-01" { + t.Errorf("article = %+v", a) + } + if page.Articles[1].Journal != "" { + t.Errorf("no journal/venue should be empty, got %q", page.Articles[1].Journal) + } + if _, err := parsePapersPage([]byte("{")); err == nil { + t.Error("bad JSON should error") + } +} + +func TestSortArticles(t *testing.T) { + as := []Article{ + {Title: "b", Year: 2020, Date: "2020-01-01", Citations: 5}, + {Title: "c", Year: 2021, Date: "2021-01-01", Citations: 1}, + {Title: "a", Year: 2020, Date: "2020-05-01", Citations: 2}, + {Title: "d", Year: 2020, Date: "2020-05-01", Citations: 9}, + } + sortArticles(as) + got := as[0].Title + as[1].Title + as[2].Title + as[3].Title + if got != "cdab" { + t.Errorf("order = %s, want cdab (year desc, date desc, citations desc)", got) + } +} + +func TestRenderArticlesHTML(t *testing.T) { + if got := renderArticlesHTML(nil); got != "

No publications found.

" { + t.Errorf("empty = %q", got) + } + page, _ := parsePapersPage([]byte(pageJSON)) + html := renderArticlesHTML(page.Articles) + for _, want := range []string{`href="https://www.semanticscholar.org/paper/p1"`, "Old & Cited", "A. One, B. Two", "2019 · J. A · 40 citations", "<Newest>", "2024 · 1 citations"} { + if !strings.Contains(html, want) { + t.Errorf("missing %q in\n%s", want, html) + } + } + if strings.Contains(html, "javascript:") { + t.Error("unsafe URL must not be linked") + } + if safeHref("ftp://x") != "" || safeHref("https://ok.test/a?b=1") != "https://ok.test/a?b=1" { + t.Error("safeHref rules") + } +} + +func TestCacheFreshness(t *testing.T) { + now := time.Date(2026, 9, 19, 12, 0, 0, 0, time.UTC) + c := cachedArticles{FetchedAt: now.Add(-2 * time.Hour)} + if !c.fresh(now, 24) || c.fresh(now, 1) { + t.Error("freshness by cache_hours") + } + if (cachedArticles{}).fresh(now, 24) { + t.Error("zero FetchedAt is never fresh") + } + if parseIntSetting("", 50, 0) != 50 || parseIntSetting("7", 50, 0) != 7 || parseIntSetting("x", 50, 0) != 50 || parseIntSetting("0", 50, 0) != 50 { + t.Error("parseIntSetting") + } + if parseIntSetting("9999", 50, maxArticleLimit) != 500 || parseIntSetting("500", 50, maxArticleLimit) != 500 || parseIntSetting("9999", 24, 0) != 9999 { + t.Error("parseIntSetting clamp") + } +} +``` + +`fetch_test.go`: +```go +package main + +import ( + "errors" + "strings" + "testing" +) + +func TestFetchAll(t *testing.T) { + calls := []string{} + get := func(url string, headers map[string]string) (int, []byte, error) { + calls = append(calls, url) + if headers["x-api-key"] != "k" { + t.Errorf("api key header missing: %v", headers) + } + if strings.Contains(url, "offset=0") { + return 200, []byte(`{"offset":0,"next":2,"data":[{"title":"a","year":1},{"title":"b","year":2}]}`), nil + } + return 200, []byte(`{"offset":2,"next":null,"data":[{"title":"c","year":3}]}`), nil + } + as, err := fetchAll(get, "123", "k", 10) + if err != nil || len(as) != 3 { + t.Fatalf("got %d articles, %v", len(as), err) + } + if len(calls) != 2 || !strings.HasPrefix(calls[0], "https://api.semanticscholar.org/graph/v1/author/123/papers?") || !strings.Contains(calls[0], "limit=100") || !strings.Contains(calls[1], "offset=2") { + t.Errorf("calls = %v", calls) + } + // limit keeps the newest N: every page is still read, then the sorted + // result is truncated + calls = nil + as, _ = fetchAll(get, "123", "k", 1) + if len(as) != 1 || len(calls) != 2 || as[0].Title != "c" { + t.Errorf("limit=1: %d articles, %d calls, first %+v", len(as), len(calls), as) + } + // non-200 → error + bad := func(string, map[string]string) (int, []byte, error) { return 429, []byte("slow down"), nil } + if _, err := fetchAll(bad, "123", "", 5); err == nil || !strings.Contains(err.Error(), "429") { + t.Errorf("429 should error with status, got %v", err) + } + failing := func(string, map[string]string) (int, []byte, error) { return 0, nil, errors.New("net down") } + if _, err := fetchAll(failing, "123", "", 5); err == nil { + t.Error("transport error should propagate") + } + if _, err := fetchAll(get, "", "", 5); err == nil { + t.Error("empty id should error") + } +} +``` + +`main_native.go`: +```go +//go:build !wasip1 + +package main + +func main() {} +``` + +Run: `GOTOOLCHAIN=go1.26.1 go test ./...` → FAIL to compile. + +- [ ] **Step 3: Implement** `article.go`: + +```go +package main + +import ( + "encoding/json" + "html" + "net/url" + "sort" + "strconv" + "strings" + "time" +) + +// Article is one publication as rendered on the Research page. +type Article struct { + Title string `json:"title"` + Authors string `json:"authors"` + URL string `json:"url"` + Year int `json:"year"` + Date string `json:"date"` // publicationDate, YYYY-MM-DD when known + Journal string `json:"journal"` + Citations int `json:"citations"` +} + +type papersPage struct { + Next *int + Articles []Article +} + +// parsePapersPage decodes one page of /author/{id}/papers. +func parsePapersPage(b []byte) (papersPage, error) { + var raw struct { + Next *int `json:"next"` + Data []struct { + URL string `json:"url"` + Title string `json:"title"` + Year int `json:"year"` + PublicationDate string `json:"publicationDate"` + Venue string `json:"venue"` + CitationCount int `json:"citationCount"` + Authors []struct { + Name string `json:"name"` + } `json:"authors"` + Journal *struct { + Name string `json:"name"` + } `json:"journal"` + } `json:"data"` + } + if err := json.Unmarshal(b, &raw); err != nil { + return papersPage{}, err + } + page := papersPage{Next: raw.Next} + for _, p := range raw.Data { + names := make([]string, 0, len(p.Authors)) + for _, a := range p.Authors { + if a.Name != "" { + names = append(names, a.Name) + } + } + journal := p.Venue + if p.Journal != nil && p.Journal.Name != "" { + journal = p.Journal.Name + } + page.Articles = append(page.Articles, Article{ + Title: p.Title, Authors: strings.Join(names, ", "), URL: p.URL, Year: p.Year, + Date: p.PublicationDate, Journal: journal, Citations: p.CitationCount, + }) + } + return page, nil +} + +// sortArticles orders newest first: year, then publication date, then citations. +func sortArticles(as []Article) { + sort.SliceStable(as, func(i, j int) bool { + if as[i].Year != as[j].Year { + return as[i].Year > as[j].Year + } + if as[i].Date != as[j].Date { + return as[i].Date > as[j].Date + } + return as[i].Citations > as[j].Citations + }) +} + +// safeHref allows only absolute http(s) URLs into href attributes. +func safeHref(raw string) string { + u, err := url.Parse(raw) + if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" { + return "" + } + return html.EscapeString(u.String()) +} + +// renderArticlesHTML is the same markup goblog's compiled-in scholar plugin produced. +func renderArticlesHTML(articles []Article) string { + if len(articles) == 0 { + return `

No publications found.

` + } + var b strings.Builder + for _, a := range articles { + b.WriteString(`
`) + if href := safeHref(a.URL); href != "" { + b.WriteString(``) + } else { + b.WriteString(`
` + html.EscapeString(a.Title) + `
`) + } + if a.Authors != "" { + b.WriteString(`
` + html.EscapeString(a.Authors) + `
`) + } + var meta []string + if a.Year > 0 { + meta = append(meta, strconv.Itoa(a.Year)) + } + if a.Journal != "" { + meta = append(meta, html.EscapeString(a.Journal)) + } + if a.Citations > 0 { + meta = append(meta, strconv.Itoa(a.Citations)+" citations") + } + if len(meta) > 0 { + b.WriteString(`
` + strings.Join(meta, " · ") + `
`) + } + b.WriteString(`
`) + } + return b.String() +} + +// cachedArticles is what the plugin keeps under the store key "articles". +type cachedArticles struct { + FetchedAt time.Time `json:"fetched_at"` + Articles []Article `json:"articles"` +} + +func (c cachedArticles) fresh(now time.Time, cacheHours int) bool { + if c.FetchedAt.IsZero() { + return false + } + return now.Sub(c.FetchedAt) < time.Duration(cacheHours)*time.Hour +} + +// maxArticleLimit caps article_limit: more than this is never useful on a +// single page, and it keeps fetchAll's page bound meaningful. +const maxArticleLimit = 500 + +// parseIntSetting reads a positive integer setting with a default, clamped +// to max when max > 0. +func parseIntSetting(s string, def, max int) int { + n, err := strconv.Atoi(strings.TrimSpace(s)) + if err != nil || n <= 0 { + return def + } + if max > 0 && n > max { + return max + } + return n +} + +const ( + unavailableHTML = `` + noIDHTML = `` +) +``` + +`fetch.go`: +```go +package main + +import ( + "errors" + "fmt" + "net/url" + "strconv" +) + +const ( + apiBase = "https://api.semanticscholar.org/graph/v1" + fields = "title,authors,year,publicationDate,venue,journal,citationCount,url" + pageSize = 100 +) + +// getter performs an HTTP GET; the wasm build uses the PDK, tests inject a fake. +type getter func(url string, headers map[string]string) (status int, body []byte, err error) + +// maxPages bounds pagination. article_limit is clamped to maxArticleLimit +// (500), so maxPages*pageSize = 1000 is the most the fetch ever +// needs; an author with more papers than that gets the newest 1000 sorted +// and truncated like everyone else. +const maxPages = 10 + +// fetchAll pages through an author's papers, sorts them newest first, and +// keeps the first limit entries. All pages are read before truncating so +// "limit" means the newest N, not the API's first N. +func fetchAll(get getter, authorID, apiKey string, limit int) ([]Article, error) { + if authorID == "" { + return nil, errors.New("semantic scholar author id is empty") + } + headers := map[string]string{"Accept": "application/json"} + if apiKey != "" { + headers["x-api-key"] = apiKey + } + var out []Article + offset := 0 + for page := 0; page < maxPages; page++ { + u := fmt.Sprintf("%s/author/%s/papers?fields=%s&limit=%d&offset=%d", apiBase, url.PathEscape(authorID), fields, pageSize, offset) + status, body, err := get(u, headers) + if err != nil { + return nil, err + } + if status != 200 { + return nil, fmt.Errorf("semantic scholar returned HTTP %d: %s", status, truncate(string(body), 200)) + } + p, err := parsePapersPage(body) + if err != nil { + return nil, fmt.Errorf("decode semantic scholar response: %w", err) + } + out = append(out, p.Articles...) + if p.Next == nil || len(p.Articles) == 0 { + break + } + offset = *p.Next + } + sortArticles(out) + if len(out) > limit { + out = out[:limit] + } + return out, nil +} + +func truncate(s string, n int) string { + if len(s) <= n { + return s + } + return s[:n] + "…" +} +``` + +In `TestFetchAll`, the `limit=1` case therefore expects `len(as) == 1` **and** `len(calls) == 2` (both pages are read before truncating), and the kept article is the newest (`year 3`, title `c`) — assert `as[0].Title == "c"`. + +Run: `GOTOOLCHAIN=go1.26.1 go test ./...` → PASS (native). + +- [ ] **Step 4: Commit** — `git add -A && git commit -m "Scholar publications plugin: article model, Semantic Scholar client and rendering + +Co-Authored-By: Claude Opus 5 (1M context) "` + +--- + +### Task 2: WASM exports, packaging, PR + +- [ ] **Step 1: `main.go`** (`//go:build wasip1`): + +```go +//go:build wasip1 + +package main + +import ( + "encoding/json" + "time" + + pdk "github.com/extism/go-pdk" +) + +//go:wasmimport extism:host/user store_get +func hostStoreGet(uint64) uint64 + +//go:wasmimport extism:host/user store_set +func hostStoreSet(uint64, uint64) uint64 + +func storeGet(key string) ([]byte, bool) { + k := pdk.AllocateString(key) + defer k.Free() + off := hostStoreGet(k.Offset()) + if off == 0 { + return nil, false + } + return pdk.FindMemory(off).ReadBytes(), true +} + +func storeSet(key string, value []byte) bool { + k := pdk.AllocateString(key) + defer k.Free() + v := pdk.AllocateBytes(value) + defer v.Free() + return hostStoreSet(k.Offset(), v.Offset()) == 0 +} + +type hookInput struct { + Settings map[string]string `json:"settings"` + Request struct { + SubPath string `json:"sub_path"` + } `json:"request"` +} + +type jobInput struct { + Name string `json:"name"` + Settings map[string]string `json:"settings"` +} + +type setting struct { + Key string `json:"key"` + Type string `json:"type"` + Default string `json:"default"` + Label string `json:"label"` + Description string `json:"description"` +} + +func outputJSON(v any) int32 { + if err := pdk.OutputJSON(v); err != nil { + pdk.SetErrorString("encode output: " + err.Error()) + return 1 + } + return 0 +} + +func pdkGet(url string, headers map[string]string) (int, []byte, error) { + req := pdk.NewHTTPRequest(pdk.MethodGet, url) + for k, v := range headers { + req.SetHeader(k, v) + } + resp := req.Send() + return int(resp.Status()), resp.Body(), nil +} + +//go:wasmexport identity +func identity() int32 { + return outputJSON(map[string]string{"name": "scholar", "display_name": "Scholar Publications", "version": "2.0.0"}) +} + +//go:wasmexport settings +func settings() int32 { + return outputJSON([]setting{ + {Key: "enabled", Type: "text", Default: "false", Label: "Enabled", Description: "Set to 'true' to enable the Research page"}, + {Key: "semantic_scholar_id", Type: "text", Default: "", Label: "Semantic Scholar Author ID", Description: "The number at the end of your semanticscholar.org author URL (e.g. 1792904)"}, + {Key: "semantic_scholar_api_key", Type: "text", Default: "", Label: "Semantic Scholar API Key", Description: "Optional; raises the API rate limit"}, + {Key: "article_limit", Type: "text", Default: "50", Label: "Article Limit", Description: "Maximum number of publications to show"}, + {Key: "cache_hours", Type: "text", Default: "24", Label: "Cache Hours", Description: "How long fetched publications are reused before refreshing"}, + }) +} + +//go:wasmexport pages +func pages() int32 { + return outputJSON([]map[string]any{{"page_type": "research", "title": "Research", "slug": "research", "show_in_nav": true, "nav_order": 20, "description": "Publications from Semantic Scholar"}}) +} + +//go:wasmexport jobs +func jobs() int32 { + return outputJSON([]map[string]any{{"name": "refresh", "interval_seconds": 3600}}) +} + +// loadCache reads the stored articles; ok is false when absent or undecodable. +func loadCache() (cachedArticles, bool) { + b, ok := storeGet("articles") + if !ok { + return cachedArticles{}, false + } + var c cachedArticles + if err := json.Unmarshal(b, &c); err != nil { + return cachedArticles{}, false + } + return c, true +} + +// refresh fetches and stores when the cache is missing or stale. It returns +// the articles to render (fresh, newly fetched, or stale-as-fallback) and +// whether any are available. +func refresh(settings map[string]string, force bool) ([]Article, bool) { + id := settings["semantic_scholar_id"] + if id == "" { + return nil, false + } + hours := parseIntSetting(settings["cache_hours"], 24, 0) + limit := parseIntSetting(settings["article_limit"], 50, maxArticleLimit) + cache, have := loadCache() + if have && !force && cache.fresh(time.Now(), hours) { + return cache.Articles, true + } + articles, err := fetchAll(pdkGet, id, settings["semantic_scholar_api_key"], limit) + if err != nil { + pdk.Log(pdk.LogWarn, "semantic scholar fetch failed: "+err.Error()) + if have { + return cache.Articles, true // stale beats nothing + } + return nil, false + } + b, _ := json.Marshal(cachedArticles{FetchedAt: time.Now(), Articles: articles}) + if !storeSet("articles", b) { + pdk.Log(pdk.LogWarn, "could not store the publications cache") + } + return articles, true +} + +//go:wasmexport render_page +func renderPage() int32 { + var in hookInput + if err := json.Unmarshal(pdk.Input(), &in); err != nil { + pdk.SetErrorString("render_page: " + err.Error()) + return 1 + } + if in.Request.SubPath != "" { + return outputJSON(map[string]any{}) // no sub-pages: goblog 404s + } + if in.Settings["semantic_scholar_id"] == "" { + return outputJSON(map[string]any{"html": noIDHTML}) + } + articles, ok := refresh(in.Settings, false) + if !ok { + return outputJSON(map[string]any{"html": unavailableHTML}) + } + return outputJSON(map[string]any{"html": renderArticlesHTML(articles)}) +} + +//go:wasmexport run_job +func runJob() int32 { + var in jobInput + if err := json.Unmarshal(pdk.Input(), &in); err != nil { + pdk.SetErrorString("run_job: " + err.Error()) + return 1 + } + // goblog's wasm adapter never calls run_job while the plugin's + // `enabled` setting is off (plugin/wasm/wasm.go, ScheduledJobs), so + // only the id needs checking here. + if in.Name == "refresh" && in.Settings["semantic_scholar_id"] != "" { + refresh(in.Settings, false) // only fetches when stale + } + return outputJSON(map[string]any{}) +} + +func main() {} +``` +(`pdk.AllocateBytes` exists in go-pdk 1.1.3; if not, use `pdk.AllocateString(string(value))`.) + +- [ ] **Step 2: Packaging** + +`goblog-plugin.json`: +```json +{ + "name": "scholar", + "display_name": "Scholar Publications", + "description": "A Research page listing your publications from Semantic Scholar, with citation counts, cached and refreshed daily.", + "author": "Jason Ernst", + "license": "Apache-2.0", + "runtime": "wasm", + "entry": "plugin.wasm", + "allowed_hosts": ["api.semanticscholar.org"], + "min_goblog_version": "0.2.9", + "homepage": "https://github.com/goblogplatform/goblog-plugin-scholar" +} +``` +`.github/workflows/release.yml` — identical to hello v2's (checkout + setup-go with `go-version-file`, build with `GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared -ldflags="-s -w" -o plugin.wasm .`, `gh release upload "$TAG" plugin.wasm --clobber`, `permissions: contents: write`), plus a `test` job on `push`/`pull_request` running `go test ./...`. +`README.md`: what it does; Install (Admin → Plugins → Scholar Publications → Install; goblog ≥ 0.2.9); Settings table (the five keys; note that installs upgrading from the compiled-in plugin keep `enabled`, `semantic_scholar_id`, `semantic_scholar_api_key`, `article_limit`, and that Google Scholar/`source`/`scholar_id` are gone); "Talks to api.semanticscholar.org only"; caching (store, `cache_hours`, hourly job); build instructions; License. +`CHANGELOG.md`: `## 2.0.0 — first release as a WebAssembly plugin; replaces goblog's compiled-in scholar plugin (Semantic Scholar only).` + +- [ ] **Step 3: Build, validate, live check** + +```bash +GOTOOLCHAIN=go1.26.1 go test ./... && GOOS=wasip1 GOARCH=wasm GOTOOLCHAIN=go1.26.1 go build -buildmode=c-shared -ldflags="-s -w" -o plugin.wasm . && ls -la plugin.wasm +cd ~/dev/goblog && go run . validate-plugin ~/dev/goblog-plugin-scholar/plugin.wasm +``` +Expected `{"name":"scholar","display_name":"Scholar Publications","version":"2.0.0","runtime":"wasm"}`. + +Live check in a local goblog (fresh sqlite; `.env` with `database=sqlite`, `sqlite_db=database.db`, `SESSION_KEY`, dummy `client_id`/`client_secret`): copy `plugin.wasm` to `plugins/wasm/scholar.wasm`, write the sidecar `plugins/wasm/scholar.json` = `{"allowed_hosts":["api.semanticscholar.org"]}`, start goblog, set settings via sqlite (`plugin_settings` rows for `scholar`: `enabled=true`, `semantic_scholar_id=1792904` — Jason's id used by goblog.live's scholar settings; if unsure use any public author id such as `1741101` (Oren Etzioni)), then `curl localhost:7000/research` → publications HTML with `citations`; second request is served from the cache (check the log has one fetch); `plugin_store` has an `articles` row. Record the transcript. + +- [ ] **Step 4: Commit, push, PR** + +```bash +git add -A && git commit -m "Add WASM exports, packaging and the release workflow + +Co-Authored-By: Claude Opus 5 (1M context) " +git push -u origin feat/initial +gh pr create --base main --title "Scholar Publications as a WebAssembly plugin (v2.0.0)" --body "..." +``` +After the maintainer merges: `gh release create v2.0.0 --title v2.0.0 --notes "..."`, watch the Release workflow, confirm the `plugin.wasm` asset; then open a PR to `goblogplatform/plugins` adding ` - repo: goblogplatform/goblog-plugin-scholar` to `registry.yaml` (its `Validate` runs the real check).