Skip to content

feat(libs): vendor the extension libraries pinned to OpenSysML-Extensions-Library, with an engine contract - #881

Merged
HuiJun merged 22 commits into
developfrom
feature/pinned-extension-libraries
Oct 10, 2026
Merged

HuiJun merged 22 commits into
developfrom
feature/pinned-extension-libraries

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

What and why

The non-normative OpenSysML Libraries/ extensions are now maintained at Open-MBEE/OpenSysML-Extensions-Library, so other SysML v2 tools can consume them. That repository imported them with their history in Open-MBEE/OpenSysML-Extensions-Library#1. OpenSysML keeps a vendored copy at the same path: go install and the module proxy must build without fetching another repository, and the embed, element identity and stdlib.snapshot stay as they are. Library changes that landed on develop directly (#816, #871, #886, #920 and #966) were ported upstream in Open-MBEE/OpenSysML-Extensions-Library#2. Relative to develop, the only change to the 16 vendored files is the library package declarations described below.

  • Pin: scripts/extension-libraries-pin.sh holds EXTENSIONS_REPO, EXTENSIONS_REF and EXTENSIONS_COMMIT, currently main at 5a47e595b9abe42e7545854eab21af20c4b1a875, the merge commit of fix(libraries): declare DiagramLayout, IdentityMetadata, MigrationMetadata and SysMLValidation as library package OpenSysML-Extensions-Library#3. Syncing from it leaves every vendored file and engine-contract.json unchanged.

  • Sync: scripts/sync-extension-libraries.sh fetches the pinned commit (--depth 1), verifies its sha and rewrites the directory from upstream libraries/*.{sysml,kerml} plus engine-contract.json.

    • --ref REF syncs a branch or tag instead of the pin; --source DIR syncs a local checkout.
    • It refuses an upstream with no libraries or no engine-contract.json, before writing anything. Any file in the directory that is neither upstream nor README.md is an error.
  • Drift check: make extension-libraries-check runs the sync with --check. It runs in pr.yml's changes job, so the changed-areas filter can't skip it. CircleCI gates only main/develop merges, not PRs, so it needs no change. make extension-libraries-sync re-vendors.

  • Engine contract: engine-contract.json comes from upstream and lists the 132 qualified names the engine binds, each with a kind:

    • function, with parameters in order;
    • metadata, with the features the engine reads;
    • document, element or package;
    • a migratorEmitted flag on names the SysML v1 migrator writes into user models.

    Two tests keep the manifest and the engine in step:

    • internal/exec/runtime/engine_contract_test.go: every registry function in an extension package is a manifest function entry with identical parameters, and every manifest function in a package the registry implements is registered.
    • tests/hygiene/engine_contract_test.go: parses all non-test Go under internal/, cmd/ and tools/ and constant-folds string literals, + concatenations and package-level string consts. Every extension-qualified name it finds must be in the manifest. A manifest entry no code derives is stale, unless its package composes names under a Pkg:: prefix (DiagramLayout, DocumentQueries, OpenSysMLRenderings and StateMachines), which the derivation can't enumerate.

    Upstream CI resolves every entry against the libraries with the pinned OMG pilot. It fails a PR that removes or renames an entry or changes its parameters unless contract is bumped, which means a MAJOR release and a re-pin here.

  • Advisory upstream tracking: .github/workflows/extension-libraries-upstream.yml runs nightly and on workflow_dispatch (with an input ref, default main).

    • It syncs upstream main, not the pin, checks or regenerates the snapshot, and runs go test ./... and go test -C tools ./....
    • The step summary records the synced sha, and on failure which step failed.
    • It is not a required check, never pushes, and never opens issues.
  • Docs:

    • README.md in the vendored directory: edits go upstream.
    • New page docs/project/extension-libraries.md covering pin, sync, drift check, contract and the nightly.
    • Pointers to the upstream repository in the root README and the guide, manual and project pages that describe the libraries.
    • Changelog fragment.

Library package declarations

DiagramLayout, IdentityMetadata, MigrationMetadata and SysMLValidation now read library package instead of standard library package. They are non-normative extensions, and the keyword drew the pilot warning User library packages should not be marked as standard. The change landed upstream in Open-MBEE/OpenSysML-Extensions-Library#3 (merged) and is vendored here. It is the same four-line change as #882, which will merge cleanly with it.

  • stdlib.snapshot is regenerated for the changed bytes. No test golden changed.
  • The opensysml-libraries input digest in docs/project/pilot-differential-baseline.json was re-recorded. No count moved, because the vendored files are resolved as context and are never validated.
  • changes/unreleased/extension-libraries-non-standard.changed.md records the change, including that RDF export no longer marks these packages sysml:isStandard true.

Merge order

No upstream dependency remains: the pin is on Extensions-Library main, and the branch includes current develop.

How it was verified

  • gofmt -l . (empty), go build ./..., go vet ./..., make lint, make extension-libraries-check, make stdlib-snapshot-check, make docs-check, python3 scripts/changelog.py check, python3 scripts/check-doc-ids.py, go test -C tools ./... and go test -race -count=1 -timeout 60m ./... pass.
  • Each guardrail fails when it should (scratch edits, reverted):
    • Adding a non-test string "Simulation::Bogus" fails the hygiene test.
    • Deleting a vendored manifest entry fails the hygiene test and the drift check.
    • Renaming a registry parameter (log's base → b) fails the runtime test: registry [x b] != manifest [x base].
  • The upstream workflow's command sequence ran locally with --ref main: the sync was clean and the snapshot unchanged.
  • An upstream tree without engine-contract.json is refused and leaves the vendored directory untouched.

Checklist

  • make test and make lint pass locally (go test ./... + go vet/gofmt gates)
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (see CONTRIBUTING.md)
  • Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
  • baselines regenerated and make docs-counts run if a gate count moved (no gate count moved)
  • No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/bf0cf4d7769a49a8adfee6bfd87860f3
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/bf0cf4d7769a49a8adfee6bfd87860f3?variant=devin
Requested by: @HuiJun

devin-ai-integration Bot and others added 3 commits October 4, 2026 02:37
…sML-Extensions-Library

The extension libraries live upstream; scripts/sync-extension-libraries.sh vendors the pinned commit, --check fails CI on drift, and engine-contract.json records the qualified names the engine binds.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…act.json

Co-Authored-By: jason.han <hanhuijun@gmail.com>
… and the new library home

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration Bot and others added 2 commits October 4, 2026 02:41
…ies; name all 14 libraries in README

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…lt site

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review October 4, 2026 03:55
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 7 commits October 4, 2026 03:58
…tools module in the upstream job

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…rary main

Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

The push pause is lifted for this PR, for one push after the work below.

Merge origin/develop (2f92d69) into this branch. There are four one-hunk conflicts, all in prose or build files: Makefile, README.md, docs/guide/04-repl.md and docs/internals/architecture.md. Keep develop's side and re-apply the extension-library additions on top.

Before pushing, run each of these and confirm it passes:

  • go build ./... && go vet ./... && gofmt -l . (gofmt must print nothing)
  • make stdlib-snapshot-check
  • make docs-counts
  • python3 scripts/changelog.py check
  • python3 scripts/check-doc-ids.py
  • make docs (strict)
  • the libs and repl suites

Two Devin Review findings from 10-04 are still open on this PR. Confirm in each thread that it is addressed, so they are not left for re-review.

Push once, without force-pushing, and mark the PR ready for review when CI is green.

devin-ai-integration Bot and others added 2 commits October 9, 2026 21:50
…ension-libraries

Co-Authored-By: jason.han <hanhuijun@gmail.com>

# Conflicts:
#	Makefile
#	README.md
#	docs/internals/architecture.md
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ary main

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…tions

Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration Bot and others added 5 commits October 10, 2026 01:19
…ary package declarations

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ary main

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ension-libraries

Co-Authored-By: jason.han <hanhuijun@gmail.com>

# Conflicts:
#	internal/workspace/libs/stdlib.snapshot
@HuiJun
HuiJun merged commit b5b381d into develop Oct 10, 2026
26 checks passed
@HuiJun
HuiJun deleted the feature/pinned-extension-libraries branch October 10, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant