feat: unify CLI and add release pipeline by Villoh · Pull Request #41 · henrique-coder/perplexity-webui-scraper

Villoh · 2026-05-20T14:08:05Z

Summary

replace legacy entrypoints with unified perplexity-webui-scraper CLI and token / api / mcp subcommands
add PyInstaller-based executable release matrix and multi-arch GHCR publishing
update docs, changelog, tests, and release notes generation for new release flow

Notes

I took some freedom with the unified binary / unified CLI shape to avoid shipping multiple duplicated executables per platform.
If you want different command names, release layout, or compatibility behavior, I am open to make any needed changes.

Testing

uv run pytest
uv run ruff format --check
uv run ruff check --output-format=github
uv run ty check
npx --yes prettier --check .github/workflows/publish.yml README.md docs/getting-started.md docs/api-reference.md docs/mcp-server.md
uv build

Replace 3 legacy scripts with one 'perplexity-webui-scraper' command (token, api, mcp subcommands). Add Nuitka executable matrix (5 OS/arch), multi-arch GHCR Docker image, workflow_dispatch for testing, and CLI delegation tests. Docs migrated to unified commands.

PyInstaller 6.20: 3x faster builds, native Python 3.14 support, smaller binaries. Single --onefile command replaces verbose Nuitka-Action config. Removed nuitka/pillow/zstandard from build deps.

Replace dry_run with skip_pypi, skip_docker, and skip_gh_releases workflow inputs. Split publish side effects into separate jobs so manual runs can selectively publish Docker, PyPI, and GitHub Releases. Add target_commitish for manual GitHub Release creation and make Docker notes conditional.

Add Containerfile.mcp and publish mcp, X.Y.Z-mcp, and vX.Y.Z-mcp tags alongside the API image. Keep API latest/default unchanged and document the optional MCP container workflow.

Add macOS Intel (macos-26-intel) and Windows ARM runner targets to the executable build matrix. Also update the Unreleased changelog entries for the release workflow work.

Introduce release_mode for manual dispatch. Preview mode publishes by commit SHA (Docker tags, prerelease GH release, artifact names) while official mode publishes by version. PyPI is restricted to official mode and Docker/GH release notes adapt to preview vs official tags.

Remove preview/official release_mode complexity. Manual dispatch uses explicit version input or auto-detects from source. Skip flags remain for selective publishing control.

henrique-coder · 2026-05-22T11:26:44Z

Thank you for your contribution and effort in unifying the CLI. I will review the changes in more detail later.

Regarding binary generation: for this project, I prefer to use Nuitka instead of PyInstaller. Since Nuitka compiles the Python code directly to C, the final binary has better performance, a smaller size, and a near-zero false positive rate in antivirus software.

I will check out your branch locally, test the images from ghcr.io, and apply the migration to Nuitka along with repository typing standardization before the merge.

Copilot

Pull request overview

This PR replaces the legacy multi-entrypoint setup with a unified perplexity-webui-scraper CLI (subcommands: token, api, mcp) and adds a new release pipeline that builds PyPI artifacts, PyInstaller executables, and multi-arch GHCR images, alongside documentation updates to match the new distribution model.

Changes:

Consolidate console scripts into a single Typer-based CLI with delegated subcommands and add delegation tests.
Add a GitHub Actions publish pipeline for Python packages, multi-platform executables (PyInstaller), and multi-arch container publishing (API + optional MCP image).
Update docs/changelog/container entrypoints to use the unified command surface.

Reviewed changes

Copilot reviewed 12 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`uv.lock`	Adds build-time dependencies (PyInstaller and transitive deps) and moves `typer` into base deps.
`tests/test_cli.py`	Adds tests ensuring unified CLI help and subcommand delegation behavior.
`src/perplexity_webui_scraper/mcp/server.py`	Updates the missing-token guidance to the new unified CLI invocation.
`src/perplexity_webui_scraper/cli/__main__.py`	Introduces unified CLI with `token`/`api`/`mcp` subcommands and lazy imports.
`README.md`	Updates install/run docs to reference `perplexity-webui-scraper <subcommand>` and new container images.
`pyproject.toml`	Collapses scripts into a single entrypoint, adjusts extras/dependency groups for new CLI + build tooling.
`docs/mcp-server.md`	Updates MCP invocation examples to use unified CLI; adds optional MCP Docker image section.
`docs/getting-started.md`	Updates quickstart commands for MCP/API/token generator to unified CLI; adds MCP Docker option.
`docs/api-reference.md`	Updates API CLI docs to unified CLI and GHCR-based run instructions.
`Containerfile.mcp`	Adds a dedicated MCP container build and entrypoint for `perplexity-webui-scraper mcp`.
`Containerfile`	Updates container entrypoint to `perplexity-webui-scraper api ...`.
`CHANGELOG.md`	Documents the unified CLI and new release/publish pipeline.
`.gitignore`	Ignores PyInstaller build artifacts (`build/`, `dist/`, `*.spec`).
`.github/workflows/publish.yml`	Replaces the old publish workflow with a multi-job release pipeline (package, executables, GHCR, PyPI, GH release).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    email: Annotated[str | None, typer.Argument(help="Your Perplexity account email.")] = None,
+) -> None:
+    """Generate a Perplexity session token via email OTP or magic link."""
+    from perplexity_webui_scraper.cli.commands.get_session_token import run  # noqa: PLC0415


+) -> None:
+    """Start the OpenAI-compatible REST API server."""
+    from perplexity_webui_scraper.api.launcher import main as run_api  # noqa: PLC0415
+
+    run_api(host=host, port=port, reload=reload, log_level=log_level)
+
+
+@cli.command(name="mcp")
+def mcp() -> None:
+    """Start the MCP server."""
+    from perplexity_webui_scraper.mcp.server import main as run_mcp  # noqa: PLC0415
+
+    run_mcp()


Villoh · 2026-05-22T11:47:34Z

+          LOGINS=$(git log $PREVIOUS_TAG..HEAD --format='%H' | while read sha; do
+            curl -s "https://api.github.com/repos/${{ github.repository }}/commits/$sha" \
+              -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
+              | grep -oP '(?<="login": ")[^"]+' | head -1
+          done | sort -u | grep -v '^$' | grep -v '\[bot\]')
+


@henrique-coder Yep, this is true, I tried changing it so contributors that have the option Keep my email addresses private enabled can also appear in the Contributors section, one single request instead of one per commit should be better and faster.

Villoh · 2026-05-22T11:34:21Z

Thank you for your contribution and effort in unifying the CLI. I will review the changes in more detail later.

Regarding binary generation: for this project, I prefer to use Nuitka instead of PyInstaller. Since Nuitka compiles the Python code directly to C, the final binary has better performance, a smaller size, and a near-zero false positive rate in antivirus software.

I will check out your branch locally, test the images from ghcr.io, and apply the migration to Nuitka along with repository typing standardization before the merge.

I tried both Nuitka and PyInstaller, but with Nuitka the performance improvement at startup is practically imperceptible, and the binary is larger, so in the end I decided to go with PyInstaller.

…ormat

…c groups)

… rules

…cies and extras

henrique-coder · 2026-05-23T01:24:19Z

Thank you for your contribution and effort in unifying the CLI. I will review the changes in more detail later.
Regarding binary generation: for this project, I prefer to use Nuitka instead of PyInstaller. Since Nuitka compiles the Python code directly to C, the final binary has better performance, a smaller size, and a near-zero false positive rate in antivirus software.
I will check out your branch locally, test the images from ghcr.io, and apply the migration to Nuitka along with repository typing standardization before the merge.

I tried both Nuitka and PyInstaller, but with Nuitka the performance improvement at startup is practically imperceptible, and the binary is larger, so in the end I decided to go with PyInstaller.

Thanks for checking both out and sharing your feedback! I completely see where you're coming from regarding the initial size and startup perception, but I still want to stick with Nuitka for the release matrix. Here is why:

AV False Positives: PyInstaller is unfortunately a magnet for false positives on Windows Defender and other engines because malware authors use it heavily to pack scripts. Since Nuitka compiles the code directly to C, it behaves like standard native software, which keeps AV flags at basically zero. This saves us a lot of headaches with users opening issues thinking the scraper is a virus.
Startup & Temp Files: PyInstaller’s --onefile works by extracting the whole compressed Python environment to a temp folder (_MEIxxxxxx) every single time it runs. This hits disk I/O and leaves temp junk behind. Nuitka runs natively from the get-go with no extraction.
Size Control: We can actually get the Nuitka binary size way down. By enabling Link-Time Optimization (--lto=yes), dropping asserts/docstrings, and using zstandard compression, the final executable ends up being very competitive and clean.

I've already integrated Nuitka into the pipeline on top of the awesome CLI unification you built, so we'll roll with this setup. Thanks again for doing the heavy lifting on the CLI structure. It makes the project a whole lot better!

Villoh added 18 commits May 19, 2026 23:44

fix: use nuitka git tag 4.0.8 instead of pypi version 4.1.1

4121ce2

chore: bump upload/download-artifact to v7

480e0d1

feat: add app icon to executable builds

892542b

fix: packaging steps use build/ not build/nuitka/

dc36caf

fix: use pre-built .icns for macOS instead of runtime PNG conversion

d1373e1

migrate: Nuitka -> PyInstaller for executable builds

398d323

PyInstaller 6.20: 3x faster builds, native Python 3.14 support, smaller binaries. Single --onefile command replaces verbose Nuitka-Action config. Removed nuitka/pillow/zstandard from build deps.

chore: drop macos-13, use macos-latest (ARM64 only)

a4f5a59

feat: publish optional MCP Docker image

ce2fe65

Add Containerfile.mcp and publish mcp, X.Y.Z-mcp, and vX.Y.Z-mcp tags alongside the API image. Keep API latest/default unchanged and document the optional MCP container workflow.

chore: expand release matrix and update changelog

89a06c3

Add macOS Intel (macos-26-intel) and Windows ARM runner targets to the executable build matrix. Also update the Unreleased changelog entries for the release workflow work.

chore: drop windows arm64 executable target

6c5740c

chore: bump Docker actions to Node 24-ready versions

f955943

refactor: drop preview release mode, simplify workflow

de8ddb4

Remove preview/official release_mode complexity. Manual dispatch uses explicit version input or auto-detects from source. Skip flags remain for selective publishing control.

fix: resolve contributors from commit SHA logins instead of emails

f30e76d

fix: contributors inline left-to-right and limit commit history to 50

c14b752

fix: lowercase GHCR image refs and filter bot contributors

aebe3c4

pull-request-size Bot added the size/XL label May 20, 2026

test: stub lazy-imported CLI modules in delegation tests

a660f11

Villoh marked this pull request as ready for review May 20, 2026 14:25

henrique-coder requested a review from Copilot May 22, 2026 11:27

Copilot started reviewing on behalf of henrique-coder May 22, 2026 11:27 View session

Copilot AI reviewed May 22, 2026

View reviewed changes

henrique-coder added 4 commits May 22, 2026 20:32

build: migrate executable compiler from PyInstaller to Nuitka

dc8fcd8

chore: optimize container builds with non-editable uv sync

5aee4f9

test: refactor sys.modules mock in CLI tests

7413c8e

chore: configure taplo formatter for toml files via pnpm

7484dbb

henrique-coder added 15 commits May 22, 2026 20:56

docs: update changelog to reflect Nuitka migration

475f2ab

chore: setup prek pre-commit hooks for format and lint

ebc21e4

docs: replace docker with podman in documentation and examples

935c1b1

ci: fix missing pnpm and enforce comprehensive linting

2321d09

ci: use official pnpm/action-setup for cross-platform compatibility

a7dcb5b

ci: bump pnpm action and version

a1ebfb4

ci: exclusively use justfile commands for format and lint

8f90173

chore: optimize format task by running ruff check --fix before ruff f…

ad503e7

…ormat

ci: enforce strict dependency installation (frozen lockfile + specifi…

8b2a332

…c groups)

docs: update changelog to reflect recent CI and tooling fixes

8e9f18f

docs: remove internal tooling details from changelog

d2ad997

chore: exclude lockfiles from prettier formatting and organize ignore…

5a6ffbe

… rules

chore: restore native pnpm-lock.yaml formatting

2b44ce3

ci: resolve module imports in linting job by installing base dependen…

d3b6047

…cies and extras

chore: pin uv setup to latest version across all CI/CD workflows

d0814c8

henrique-coder merged commit 0f2b9bb into henrique-coder:prod May 23, 2026
2 checks passed

henrique-coder mentioned this pull request May 23, 2026

feat: add TOTP 2FA support to session token generation #42

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: unify CLI and add release pipeline#41

feat: unify CLI and add release pipeline#41
henrique-coder merged 38 commits into
henrique-coder:prodfrom
Villoh:feature/release-matrix-docker

Villoh commented May 20, 2026 •

edited

Loading

Uh oh!

henrique-coder commented May 22, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Villoh May 22, 2026

Uh oh!

Villoh commented May 22, 2026

Uh oh!

henrique-coder commented May 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Villoh commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Notes

Testing

Uh oh!

henrique-coder commented May 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Villoh May 22, 2026

Choose a reason for hiding this comment

Uh oh!

Villoh commented May 22, 2026

Uh oh!

henrique-coder commented May 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Villoh commented May 20, 2026 •

edited

Loading