Repository navigation
Conversation
|
|
zjncs
marked this pull request as draft
September 20, 2026 01:39
zjncs
changed the title
zjncs
force-pushed
the
feat/skill-authoring-validation
branch
2 times, most recently
from
September 22, 2026 16:17
dd9482c to
1389a77
Compare
zjncs
force-pushed
the
feat/skill-authoring-validation
branch
from
October 1, 2026 02:58
1389a77 to
3c0b2e1
Compare
This was referenced Oct 2, 2026
zjncs
marked this pull request as ready for review
October 2, 2026 17:25
Author
|
Hi @XiaoSeS, this PR is now ready for review — all checks green (DCO ✅, CLA ✅), MERGEABLE, and rebased on the latest main. It implements the Agent Skill authoring & validation platform from OSPP task 26d8e0076. Quick map to the task's deliverables: Draft authoring & validation
Fix loop & publish integration
Security (per the audit feedback): SSRF policy on OpenAI/MCP endpoints (always-blocks cloud-metadata/link-local; private ranges behind flags), stdio MCP off by default with docker isolation, envRefs allowlist, docker execution-mode default with a startup guard; response/line caps; regression tests for all of it. RISC-V64 (the task lists it as a supported architecture): repeatable real-board CI on physical VisionFive 2 / Banana Pi F3 hardware via free board runners — see the Scale: 13 commits, 150 files, +17.9k/−971; backend 1085 tests 0 failures; live smoke 39/39 ( The OSPP final review window is approaching — would you have time to take a look? Happy to address any feedback. Thanks! |
zjncs
force-pushed
the
feat/skill-authoring-validation
branch
from
October 8, 2026 08:11
d180407 to
310ac34
Compare
…istence Skill drafts with per-file content-addressed storage, runtime bindings (agent type, adapter config, tool allowlist, MCP server declarations), validation runs with ordered events and findings, plus the Flyway V60 schema (JSONB columns) and JPA repositories. Domain services cover draft lifecycle, file save/read with optimistic revision checks, binding validation, fix application, submit gating, and the structure/spec/ assertion rules with fix suggestions. Signed-off-by: zjncs <18910855655@163.com>
…d MCP probe ValidationRunOrchestrator materializes a draft into a one-shot workspace and runs structure, configuration and behavior layers, streaming ordered events and findings. The local-script runtime executes inline for dev or in a locked-down Docker container (no network, resource caps, read-only rootfs, dropped capabilities); the OpenAI-compatible runtime drives an agent loop with real MCP tool execution. Declared MCP servers are probed for real at run time — connect, initialize handshake, tools/list — and unreachable servers or unknown toolFilters become CONFIG-layer findings even when validation.yaml is absent. A maintenance task sweeps crashed runs. Signed-off-by: zjncs <18910855655@163.com>
Endpoints for the authoring workbench: draft CRUD and file management (text + binary upload, content-addressed storage), runtime binding, validation run lifecycle with SSE event streaming and polling fallback, findings with fix application and dismissal, and validated draft submission into the publish pipeline. The integration test runs the full create → edit → bind → validate → fix → revalidate → submit loop on real PostgreSQL, including the dead-MCP-server regression. Signed-off-by: zjncs <18910855655@163.com>
Draft list and detail pages with a file editor (create/edit/upload, binary files render metadata + sha256, delete with explicit reselection), runtime binding form (local-script, docker mode, OpenAI-compatible, MCP server declarations), validation run pages with a live event console (SSE with polling fallback, terminal-run backfill), findings with diff previews and two-step fix confirmation, and submit gating on the validated revision. Dev proxy target is configurable via VITE_API_PROXY_TARGET. Signed-off-by: zjncs <18910855655@163.com>
Playwright E2E drives the real UI against the real API: draft creation with scaffold regression guard, file editing across create/upload (selection integrity), runtime binding, validation to SUCCEEDED with live events, frontmatter failure → fix preview → revalidate, and binary upload verification. The smoke script exercises the live API surface (35 checks): full validation loop, fixable findings, guards, and real MCP probes against dead and fake MCP servers. Makefile dev targets gain configurable ports and proxy target. Signed-off-by: zjncs <18910855655@163.com>
…rification Design/deploy/test doc for the authoring platform (domain model, three-layer pipeline, fix loop, API, workbench, config table), a self-contained example skill exercising every assertion type, and a dated RISC-V64 verification record: linux/riscv64 image booted under QEMU, Flyway V60 applied on fresh PostgreSQL, full authoring flow driven over the API with the validation script executing inside the emulated riscv64 container (SUCCEEDED, 0 errors). Signed-off-by: zjncs <18910855655@163.com>
Server-side validation and runtime clients now share a fail-closed AuthoringSecurityPolicy: - OpenAI-compatible and http/sse MCP endpoints are resolved and checked against SSRF rules at save time and again at connect time: any-local, link-local (cloud metadata) and multicast addresses are always rejected; loopback, RFC1918, unique-local IPv6 and CGNAT are rejected unless the per-surface allow-private-endpoints flag is set; unresolvable hosts are rejected. HTTP redirects are never followed and response bodies are capped at 2 MiB on both surfaces. - stdio MCP transport is disabled by default (mcp.stdio-enabled) and, in docker execution mode, its command is wrapped in a hardened container (no network, dropped capabilities, read-only root, tmpfs, pid/memory/cpu limits) that is torn down when the client closes; stdio lines are capped at 2 MiB with force-kill on overflow. - envRefs may only name variables in mcp.env-allowlist (empty by default, so every reference is rejected until an operator opts in). - local-script execution-mode now defaults to docker; running inline outside the local/dev/test profiles fails startup. Also renumbers the authoring migration V60 -> V66 to clear the slot taken by upstream organization migrations. Signed-off-by: zjncs <18910855655@163.com>
- ConfiguredAuthoringSecurityPolicyTest: public/private classification per surface, always-blocked ranges, unresolvable hosts, no-host URIs. - AuthoringStartupGuardTest: inline mode only boots with a non-production profile; docker mode always passes. - McpClientFactoryTest: rejected endpoints never connect, disabled stdio refuses, envRefs are filtered by the allowlist, docker-wrapped commands carry the full hardening argument set. - HttpMcpClientTest/StdioMcpClientTest: redirects to the cloud metadata range are not followed, 3 MiB bodies and unterminated 5 MiB lines are rejected (the latter with the process killed), teardown commands run on close. - AuthoringFlowIntegrationTest: metadata endpoints and envRefs outside the allowlist are rejected at save time with actionable reasons. - Smoke case 5 asserts the same three rejections over HTTP. Signed-off-by: zjncs <18910855655@163.com>
- Record the 2026-09-21 end-to-end publish run: author -> validate -> submit PUBLIC -> async scan (SKILL_SCANNER:SAFE) -> admin review -> PUBLISHED, publicly retrievable, driven by the new scripts/authoring-publish-e2e.sh (16 checks, kept for re-runs). - Document the security configuration keys and baseline, the docker default for local-script execution, and the migration renumber to V66. - Add scripts/riscv64-verify.sh (native vs QEMU auto-detect) plus the native-hardware checklist; native runs remain an open item with no RISC-V hardware available to the project. Signed-off-by: zjncs <18910855655@163.com>
A fake-IP VPN resolver on the dev machine answered every lookup inside 198.18.0.0/15, and the policy let that reserved range through — a real SSRF classification gap, since the range is never a legitimate endpoint and is the synthetic pool used by fake-IP proxies. It now follows the same conditional block as loopback/RFC1918/ULA/CGNAT. The unresolvable-host test now stubs resolution through an injectable HostResolver seam instead of relying on the live network (static mocks are unavailable: the build pins the subclass mock maker), covering both a hard lookup failure and a synthetic fake-IP answer. The Spring-wired constructor is annotated @Autowired so the extra test seam constructor does not break bean creation. Signed-off-by: zjncs <18910855655@163.com>
Native riscv64 hosts could not build the server image: the Alpine JDK build stage has no riscv64 variant. The build stage base is now a BUILD_IMAGE build arg (default unchanged), with the Noble JDK variant passed on native hosts. scripts/riscv64-native-setup.sh runs on the RVLab board itself: installs Docker/PostgreSQL/Redis (apt or dnf), creates the empty database the run migrates from zero, exposes PG and Redis to the docker bridge, builds the image natively (no QEMU) and drives scripts/riscv64-verify.sh, teeing everything into a dated evidence log. The doc checklist now points at the script and records the pending RVLab access requests. Signed-off-by: zjncs <18910855655@163.com>
Adds .github/workflows/riscv64-native.yml targeting the free Cloud-V / 10xEngineers board runners: on a physical VisionFive 2 it records the board identity, installs JDK 21 (apt, with a Temurin riscv64 tarball fallback) plus PostgreSQL and Redis, builds the project with Maven on the board, and drives the full authoring flow via the verification script. The workflow is guarded to the fork — upstream has no such runner, an unguarded job would queue there forever. scripts/riscv64-verify.sh gains a jar boot mode (SKILLHUB_RISCV_BOOT_MODE=jar) that runs an already-built jar directly instead of a Docker image, which is what the board CI uses; docker mode is unchanged. Jar mode verified end-to-end locally (boot, health, full flow, 12 events, cleanup) with only the arch assertion failing on the arm64 host as designed. Signed-off-by: zjncs <18910855655@163.com>
The Cloud-V runner workflow passed on a physical VisionFive 2 (Ubuntu 24.04 riscv64, no qemu markers): native Maven build, server boot, and the full authoring flow 7/7 in ~14 minutes total. Checklist items 1 and 3 are now covered by repeatable native CI; items 4-5 (docker execution-mode rerun, browser E2E) remain for a dedicated machine, with the RVLab applications pending. Signed-off-by: zjncs <18910855655@163.com>
Extends the real-board CI with two jobs: - authoring-docker-mode-on-riscv64 (VisionFive 2, every push): same authoring flow but with SKILLHUB_AUTHORING_LOCAL_SCRIPT_MODE=docker, so validation scripts execute inside the alpine:3.20 riscv64 container — the docker isolation path now has native-hardware evidence. Verified locally first: the TOOL_RESULT event records backend=docker, exit 0. - backend-tests-on-riscv64 (Banana Pi F3, manual dispatch only): the full ./mvnw test suite on riscv64 (different board type, so it runs in parallel and never blocks the quick verification jobs; ryuk disabled as the helper image has no riscv64 variant). Also probes system browser availability — Playwright ships no riscv64 driver, so browser E2E remains on the dev machine and the probe documents availability. Signed-off-by: zjncs <18910855655@163.com>
The board run showed: jar-mode verification green again, but the docker-mode task failed within seconds (likely a workspace bind-mount path that does not exist on the host docker daemon), and the full test-suite job never started because no bpi-f3 runner ever provisioned (24h queue, then cancelled). Changes: - riscv64-verify.sh now dumps the run's findings when the validation does not succeed, so failures are diagnosable from the job log alone; the script task timeout rises to 180s to absorb first-time image pulls in docker execution-mode. - the docker-mode job gains a pre-flight step: docker version, alpine pull, and a bind-mount round-trip that makes the runner's mount semantics explicit before the timed task runs. - the full test-suite job moves from banana-pi-f3 to visionfive2 (the pool that actually provisions) and pre-pulls the Testcontainers images so pulls stay out of the test timeouts. Signed-off-by: zjncs <18910855655@163.com>
…tics The board pre-flight proved the runner's docker daemon resolves bind-mount sources against the HOST filesystem, where the job workspace does not exist — docker execution-mode (which bind-mounts the workspace) cannot run on that runner class. The pre-flight now records the probe outcome and the verification step only runs when a bind-mount actually round-trips, so the job stays green while the limitation is explicit in the log. docs/26 states the same honestly: item 4's evidence is the local-Docker run (backend=docker in the event stream) plus the real container isolation test. Signed-off-by: zjncs <18910855655@163.com>
The first full backend-suite run on a real riscv64 board passed 1081 of 1085 tests; the only four failures were DockerScriptRuntimeAdapterTest, broken by the shared-daemon bind-mount limitation of that runner class (documented with the CI pre-flight). The test now round-trips a bind mount in @BeforeAll and skips with a clear assumption message when the environment cannot support workspace mounts — the same pattern as the existing no-docker skip. On real Docker the tests still run in full (verified locally, 4/4). The board result is recorded in docs/26: architecture-wise the suite is green on real riscv64 hardware; the environment limitation is the only delta. Signed-off-by: zjncs <18910855655@163.com>
…n run The suite itself passed on the board (BUILD SUCCESS, 1081 tests, 0 failures, 39m40s — the bind-mount assumption now skips the four docker adapter tests gracefully). The job still came up red because the runner's post-job checkout cleanup failed, most likely from disk pressure after the Maven build. A final always()-step frees the build artifacts and dangling images before the job ends; docs/26 records the green suite result and explains the red-job artifact. Signed-off-by: zjncs <18910855655@163.com>
The rerun with the disk-cleanup step finished all three jobs green: jar-mode authoring verification, the conditional docker-mode job, and the full backend suite (BUILD SUCCESS, 1081 tests, 0 failures, 38m42s). docs/26 now references the green run instead of the red-checkmark artifact. Signed-off-by: zjncs <18910855655@163.com>
zjncs
force-pushed
the
feat/skill-authoring-validation
branch
from
October 9, 2026 02:42
fc3b6bf to
a34c195
Compare
zjncs
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
面向 SkillHub 的 Agent Skill 创作与验证平台（开源之夏 26d8e0076）。作者在浏览器工作台里完成 Skill 的创建、文件编辑、运行时绑定与三层验证，验证通过的修订号过闸后一键进入既有发布管线 —— 把"写一个 Skill"从盲写 zip 包变成带真实反馈的闭环。
分层提交
feat(authoring)domain + persistencefeat(authoring)validation pipelinefeat(authoring)REST APIfeat(web)workbenchtest(authoring)docs(authoring)核心闭环
MCP_CONNECT_FAILED；toolFilters 引用未知工具 → 告警）；验证记录
mvnw test全量 1039 tests, 0 failures（Testcontainers 真实 PostgreSQL，含AuthoringFlowIntegrationTest全流程与死 MCP 服务器回归用例）scripts/authoring-smoke-test.sh35/35（含 MCP 死服务器失败、假服务器工具发现、未知 toolFilters 告警三连）linux/riscv64镜像（293MB，eclipse-temurin:21-jre-noble），QEMU 下 91.9s 启动、Flyway 全量迁移，REST API 走完 建草稿 → 编辑 → 绑定 → 验证 SUCCEEDED（0 错 0 警，12 事件），脚本子进程真实运行在 riscv64 用户态 —— 详见docs/26-skill-authoring-platform.md的验证记录章节E2E 与冒烟共暴露并修复了 5 个真实缺陷（脚手架内容未持久化、无绑定时表单默认值被清空、文件编辑器陈旧列表劫持选中导致内容存错文件、终态运行事件不回填、MCP 探针被配置层提前返回跳过）。
部署
Flyway V60 自动建表；
SKILLHUB_AUTHORING_LOCAL_SCRIPT_MODE=docker启用容器隔离（生产建议）；LLM 运行时默认关闭。配置项详见docs/26-skill-authoring-platform.md。