Skip to content

feat(subos): a subos describes itself, and packages can declare its environment (2026.8.5.1) - #480

Merged
Sunrisepeak merged 4 commits into
mainfrom
feat/subos-env-configuration-layer
Aug 4, 2026
Merged

feat(subos): a subos describes itself, and packages can declare its environment (2026.8.5.1)#480
Sunrisepeak merged 4 commits into
mainfrom
feat/subos-env-configuration-layer

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

The gap

A program needs three things to run:

layer what it is xlings had
bootstrap loader + CRT + libc ✅ glibc + elfpatch
discovery PATH + RPATH ✅ xvm + shims
configuration the env its subsystems read ❌ nothing

That third row is mcpp-community/mcpp#352: a GLFW binary that links correctly and exits 255. A GL driver is found through LIBGL_DRIVERS_PATH, an EGL vendor through __EGL_VENDOR_LIBRARY_DIRS, a font config through XDG_DATA_DIRS. None can be linked in — and the process that has to see them is the user's own binary, which xlings never wraps, so the per-shim envs on xvm.add cannot reach it.

What lands

A subos now carries a subos_info block in its own .xlings.json, saying what it is next to the workspace that says what it has:

"subos_info": {
  "schema_version": 1,
  "runtime": "glibc@2.39",
  "envs": {
    "compat.mesa@25.0.0": [
      { "var": "LIBGL_DRIVERS_PATH", "op": "set",     "value": "${pkgdir}/lib/dri" },
      { "var": "XDG_DATA_DIRS",      "op": "prepend", "value": "${pkgdir}/share" }
    ]
  },
  "created_at": "", "created_by": ""
}
  • runtime is self-describing (glibc@2.39 is Linux/glibc) and settable with xlings subos new --runtime. The family is derived, never stored, so it cannot contradict the runtime it came from.
  • envs is keyed by the declaring package's binding — the same provider-scoped ownership xvm.add uses. Uninstall drops exactly what a package added; a recipe writes no cleanup.
  • Packages declare through subos.env{} (libxpkg 0.0.48). subos use / --shell / --cmd expand and apply them, and report what was injected on stderr.
  • A variable the user already exported wins over set; prepend still composes with it.

Values must use placeholders (${pkgdir} ${subosdir} ${home} ${xlings_home}). A value holding this machine's absolute paths describes this machine, and a subos description that only works where it was written is not a description. An unresolvable placeholder is left verbatim rather than blanked — ${pkgdir}/lib collapsing to /lib is a real host path a driver search would follow.

Two design decisions changed by building it

Conflicts resolve by binding order, not install order. The design said install order. That order is not in the manifest, and recording it would add a field whose only effect is to make the outcome depend on history — two machines holding byte-identical manifests would export different values, which contradicts the whole point of a portable description. Pinned by DoesNotDependOnDeclarationOrder.

The probe is type(subos.env) == "function", not if subos.env then. subos.env arrives as a new module, and import() answers an unknown module with a permissive proxy whose every key is truthy — so the obvious probe is true on clients that will accept the call and discard it. if xvm.files then was only ever safe because xvm is a module those clients already ship. E2E-61 pins both readings against a real released binary.

Three things the design did not anticipate

  • self init writes the block too. default is not created through subos::create, so without this the entire layer would be inert on the one subos everybody actually uses — and inert is indistinguishable from "no package needed anything". It doubles as the migration for older homes.
  • Env ops are consumed before process_xvm_operations_'s early return. A package that declares only env registers nothing with xvm, and would otherwise install cleanly with its declarations dropped on the floor.
  • doctor's renderer had a default: break;. A new FindingKind without a case is detected and never printed.

doctor

Five checks — structure, sections owned by packages that are not installed, values that would not expand, contested variables, a declared runtime that is missing. Reporter and repairer call one predicate, so --fix touches exactly what was reported (and refuses to rewrite a .xlings.json it could not parse).

Verification

  • 29 unit tests: schema, invariants, expansion, conflict rules, order-independence
  • E2E-60 — install → declaration recorded (and still portable) → idempotent reinstall → --shell--cmd → user override → doctor clean → orphan detected and repaired → uninstall drops the section, envs stays {} → variable no longer injected
  • E2E-61 — one recipe through a real released 2026.8.4.2 and this build: truthy=true typed=false branch=legacy vs truthy=true typed=true branch=subos.env, and only the client that can apply a declaration recorded one

Ecosystem

repo
openxlings/libxpkg#32 subos.env + subos_env op — merged, tagged 0.0.48, mirrored to gitcode
mcpplibs/mcpp-index#153 0.0.48 in all three platform blocks — merged
openxlings/xim-pkgindex#496 V2 spec: the API and the probe rule

Design: .agents/docs/2026-08-05-subos-minimum-design.md
Plan: .agents/docs/2026-08-05-subos-slice1-landing-plan.md

Not in scope (per the design's §9.4): the compat.mesa payload itself, multiple concurrent runtimes, and the platform abstraction.

…nvironment (2026.8.5.1)

A program needs three things to run: a loader and a libc (bootstrap), a way
to find its binaries and libraries (discovery), and the environment its
subsystems look at (configuration). xlings had the first two -- glibc plus
elfpatch, xvm plus shims -- and nothing for the third.

That gap is mcpp-community/mcpp#352: a GLFW binary that links correctly and
exits 255, because a GL driver is found through LIBGL_DRIVERS_PATH, an EGL
vendor through __EGL_VENDOR_LIBRARY_DIRS, and a font config through
XDG_DATA_DIRS. None of those can be linked in, and the process that has to
see them is the user's own binary -- which xlings never wraps, so the
per-shim `envs` on xvm.add cannot reach it.

A subos now carries a `subos_info` block in its own .xlings.json saying what
it IS, next to the `workspace` that says what it HAS: which runtime its
binaries were built against, and which variables its processes need.

  subos_info: { schema_version, runtime, envs, created_at, created_by }

`runtime` is self-describing ("glibc@2.39" is Linux/glibc) and settable with
`xlings subos new --runtime`; the family is derived, never stored, so it
cannot contradict the runtime it came from. `envs` is keyed by the declaring
package's binding -- the same provider-scoped ownership xvm.add uses -- so
uninstall drops exactly what a package added and a recipe writes no cleanup.

Values must use ${pkgdir} / ${subosdir} / ${home} / ${xlings_home}. A value
holding this machine's absolute paths describes this machine, and a subos
description that only works where it was written is not a description. An
unresolvable placeholder is left verbatim rather than blanked: "${pkgdir}/lib"
collapsing to "/lib" is a real host path a driver search would follow.

Packages declare through `subos.env{}` (libxpkg 0.0.48). Entering the subos --
`subos use`, `--shell`, `--cmd` -- expands and applies them, and reports what
it injected on stderr. A variable the user already exported wins over `set`;
`prepend` still composes with it.

Conflicts resolve by binding order, not install order. Install order is not in
the manifest, and recording it would add a field whose only effect is to make
the result depend on history: two machines holding identical manifests would
export different values. doctor reports every conflict rather than resolving
it quietly.

Three things this needed that the design did not anticipate:

  * `self init` writes the block too. The `default` subos is not created
    through subos::create, so without this the whole layer would be inert on
    the one subos everybody actually uses -- and inert is indistinguishable
    from "no package needed anything". It doubles as the migration for homes
    that predate the block.
  * env ops are consumed BEFORE process_xvm_operations_'s early return. A
    package that declares only env registers nothing with xvm, and would
    otherwise install cleanly with its declarations dropped.
  * doctor's renderer had a `default: break;`. A new FindingKind without a
    case is detected and never printed.

doctor gains five checks: structure, sections owned by packages that are not
installed, values that would not expand, contested variables, and a declared
runtime that is missing. Reporter and repairer call one predicate, so `--fix`
touches exactly what was reported.

Verified: 29 unit tests over schema, invariants, expansion, conflicts and
order-independence; E2E-60 walks install → declaration → --shell → --cmd →
user override → doctor → uninstall; E2E-61 runs one recipe through a real
released binary and this build.

E2E-61 also pins the probe rule, which the design got wrong. `subos.env`
arrives as a NEW MODULE, and import() answers an unknown module with a
permissive proxy whose every key is truthy -- so `if subos.env then` is true
on clients that will accept the call and discard it. The rule is
`type(subos.env) == "function"`; `if xvm.files then` was only ever safe
because `xvm` is a module those clients already ship.

Design: .agents/docs/2026-08-05-subos-minimum-design.md
Plan:   .agents/docs/2026-08-05-subos-slice1-landing-plan.md
The mcpp cache is keyed on hashFiles(mcpp.toml, mcpp.lock, .xlings.json) with
a `mcpp-<os>-` fallback. Changing a dependency misses the exact key, the
fallback restores BMIs built against the previous dependency set, and the
build fails with

    mcpplibs.tinyhttps: error: import 'std' has CRC mismatch

which reads like a compiler bug and is a cache restored for the wrong inputs.
The existing retry does not help -- it was written for a stale index entry,
and re-running the same build against the same cache fails identically.

Bumping only the version field never triggered it (PR #479 passed); bumping a
dependency does. Measured here on mcpplibs.xpkg 0.0.47 -> 0.0.48.

Drops only ~/.mcpp/{bmi,build-cache} on an inexact restore. The payload store
under registry/data/xpkgs is dependency-independent and stays -- it is what
makes restoring the ~800 MB cache worth doing.
Generated from cli/spec.cppm; the new flag has to appear there or
test_generated_command_reference fails the build.
The guard added in the previous commit only fires on an INEXACT restore, and
by the time it existed the damage was already stored: the first run of this
PR restored a fallback BMI set, failed, and saved that state under its own
exact key. Every later run then got an exact hit on it, skipped the guard,
and failed identically.

Retiring the key prefix is what discards those entries. The guard is what
keeps a poisoned set from being written again -- an inexact restore now drops
its BMIs before anything is built, so what gets saved is internally
consistent. Payload reuse across dependency changes is unaffected, which is
the reason for keeping restore-keys at all.

The first run on each platform after this is cold.
@Sunrisepeak
Sunrisepeak merged commit db728e1 into main Aug 4, 2026
8 checks passed
@Sunrisepeak
Sunrisepeak deleted the feat/subos-env-configuration-layer branch August 4, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant