Astrid-owned I/O: replace wasi:io with astrid:io@1.0.0 (error / poll / streams)#8
Merged
Merged
Conversation
One-time pre-adoption amendment of the @1.0.0 freeze. The WIT lint will flag this as a forbidden modification of a frozen file — the override is justified because nothing has been built against the @1.0.0 contracts yet (kernel scaffolding just landed, SDK and capsules have not migrated). Treating this as a draft correction rather than triggering the dual-version evolution path costs no downstream consumer and avoids dual-binding ceremony in every toolchain forever. What changes: - New: host/io@1.0.0.wit with the `pollable` resource and `poll(list<borrow<pollable>>) -> result<list<u32>, error-code>`. Doc spells out the audit / per-principal-quota / cancel-token guarantees that make this Astrid-owned rather than wasi-owned. - host/ipc, host/net, host/http, host/process: `use` clause now imports `pollable` from `astrid:io/poll@1.0.0` instead of `wasi:io/poll@0.2.0`. Doc comments referencing `wasi:io/poll.poll` for composition updated to `astrid:io/poll.poll`. - host/sys: header rewritten — Astrid does not expose ANY `wasi:*` to capsules. The host ABI is fully Astrid-owned for two reasons: (1) every host call must be gated/audited/principal-scoped uniformly, no carve-outs for "foundation types"; (2) the unikernel target (hermit-rs) needs the WIT contract stable across host implementations — dispatching to hermit syscalls when the kernel is the unikernel itself. - README: `astrid:io@1.0.0` row added. The wit immutability rule remains in force from this commit forward. After this PR merges, the @1.0.0 files are frozen for real. Future shape changes ship as @1.1.0 alongside the @1.0.0 files.
With the wasi:io/poll dependency replaced by astrid:io/poll@1.0.0,
no Astrid host package references wasi:* anymore. The vendored
deps/wasi-io/ tree is dead weight — remove it.
- deps/wasi-io/{error,poll,streams,world}.wit: removed.
- scripts/validate-wit.sh: header + comments updated; deps/ is now
reserved for any future vendored external packages but intentionally
empty today. The staging loop already gracefully handles an absent
deps/ directory, so no logic change.
- .github/workflows/lint.yml: comment updated to reflect the
self-contained host ABI.
Validation: `scripts/validate-wit.sh` still passes on all 13 host
files (approval, elicit, fs, guest, http, identity, io, ipc, kv, net,
process, sys, uplink).
…s into net/http/process
Mirrors wasi:io@0.2.0's three-interface foundation (error / poll /
streams) but Astrid-owned: every read / write / skip / splice is
audited per-principal, every blocking variant races against the
calling capsule's cancellation token, and stream-handle counts are
bounded by the per-principal quota profile. No wasi:* carve-outs.
This is the second amendment to the @1.0.0 freeze in the same
pre-adoption window. Nothing has been built against the @1.0.0
contracts yet; after this PR merges the @1.0.0 files are locked for
real and shape changes ship as @1.1.0 alongside.
host/io@1.0.0.wit:
- New interface 'error' with a downcastable error resource (single
to-debug-string method). Carried by stream-error::last-operation-failed.
- Existing interface 'poll' kept verbatim.
- New interface 'streams' with input-stream and output-stream resources.
Surface matches wasi:io/streams: read / blocking-read / skip /
blocking-skip / subscribe on input; check-write / write /
blocking-write-and-flush / flush / blocking-flush / write-zeroes /
blocking-write-zeroes-and-flush / splice / blocking-splice /
subscribe on output. Error variant is stream-error { closed,
last-operation-failed(error) }.
Stream accessors wired where they belong:
- host/net@1.0.0.wit: tcp-stream gains read-stream / write-stream so
proxy / forwarder capsules can pipe bytes host-side via
output-stream.splice without crossing the WASM boundary per byte.
The primary throughput primitive for capsule-hosted TCP servers.
- host/http@1.0.0.wit: http-stream gains body-stream for streaming
response bodies through splice (e.g. forwarding upstream HTTP into
a capsule-served TCP connection).
- host/process@1.0.0.wit: process-handle gains stdin / stdout /
stderr returning output-stream / input-stream / input-stream. Use
these instead of read-logs / write-stdin for high-throughput child
stdio (e.g. wrapping ffmpeg, piping shell pipelines).
Each amended @1.0.0 file gains a 'use astrid:io/streams@1.0.0.{...}'
declaration to bring the resource types into scope. The byte-oriented
methods (read-bytes / write-bytes / read-chunk / read-logs /
write-stdin) stay alongside the new stream halves; capsules pick the
access pattern that fits the use case.
UDP intentionally stays datagram-only — no stream halves. Mapping
datagrams to a byte stream loses message boundaries.
Validation: `scripts/validate-wit.sh` passes on all 13 host
packages.
…ktop-only
Reconsidered the stdin/stdout/stderr stream accessors I added in the
prior commit. They do not pull weight:
- The IPC bus already provides high-throughput capsule-to-capsule
byte movement. If capsule A spawns a child and capsule B wants its
output, A drains via read-logs and republishes on the bus; the bus
is the throughput primitive, not process stdio.
- write-stdin + read-logs already cover all realistic child stdio
patterns: REPL-style MCP stdio JSON-RPC, structured tool capture,
periodic log drain.
- The unikernel target (hermit-rs) has no POSIX fork/exec model, so
the entire process package is desktop-kernel only — forward-compat
for unikernel is moot here.
Net effect: process-handle becomes simpler (three fewer methods) and
the splice-into-child media-gateway scenario, if it ever materialises,
ships as process-handle@1.1.0 alongside the @1.0.0 file.
What changes:
- host/process@1.0.0.wit: removed stdin / stdout / stderr methods on
process-handle; removed the now-unused
'use astrid:io/streams@1.0.0.{input-stream, output-stream}' clause.
- Header doc updated to call out the desktop-only scope explicitly so
capsule authors targeting the unikernel know not to import this
package.
tcp-stream and http-stream stream halves stay — splicing TCP-to-TCP
(capsule-hosted proxies) and HTTP-body-to-sink remain real, recurring
patterns that work on the unikernel target too.
There was a problem hiding this comment.
Code Review
This pull request replaces the external wasi:io dependency with a native astrid:io implementation to ensure all I/O operations are audited, principal-scoped, and quota-bounded within the Astrid kernel. The changes include the introduction of the astrid:io WIT package and updates to http, net, ipc, and process interfaces to utilize these new primitives. Feedback focuses on several restrictive limits in the new I/O implementation: the 64-pollable limit is noted as inconsistent with IPC quotas, the 1 MiB trap on reads is flagged as non-standard, and the 4 KiB caps on blocking write/flush operations are identified as unnecessarily restrictive and potentially inefficient.
Addresses the four points raised on PR #8 and retires the frozen-file CI check while the @1.0.0 contracts are still in pre-adoption (no SDK or capsule has bound to them yet). WIT shape changes (host/io@1.0.0.wit): - poll(): per-call cap raised 64 -> 256. The 64 number was inconsistent with the per-capsule subscription quotas elsewhere (ipc allows 128 subscriptions alone); a capsule at full IPC quota plus TCP / UDP / HTTP / process pollables couldn't wait on them all in a single call. 256 covers the realistic worst case with headroom; per-principal profiles may still cap it lower. - input-stream.read(): no longer claims to trap on > 1 MiB. wasi:io semantics only trap on lengths exceeding wasm32's addressable range. The 1 MiB ceiling is a host-internal buffer ceiling that truncates the return — callers loop. Truncation is not an error and the doc now says so. - output-stream.blocking-write-and-flush(): drops the 4 KiB cap. wasi:io's 4096 is documentation pseudocode, not a contract. Real impls loop internally for arbitrary sizes; capping forces the guest into an outer loop that defeats the convenience method. Host now segments internally and yields between chunks. - output-stream.blocking-write-zeroes-and-flush(): same — drops the 4 KiB cap. This one is especially silly because the input is just a u64 length, no guest-side buffer to bound. CI / discipline: - scripts/lint-wit-immutability.sh removed. - .github/workflows/lint.yml: 'WIT frozen-file immutability' job deleted; 'WIT files parse' job kept. - README: 'Evolution discipline' section updated. The frozen-file rule remains documented; the automated check is retired during pre-adoption (no consumer means no real breakage to prevent). Re-enable from git history once a downstream consumer ships against a versioned file. scripts/validate-wit.sh passes on all 13 packages.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the
wasi:iodependency in the host ABI with an Astrid-ownedastrid:io@1.0.0package (three interfaces:error,poll,streams). The host ABI is now fully self-contained — nowasi:*surface anywhere — so every host call routes through Astrid's audit, capability, cancellation, and per-principal quota layers without exception. This is also what makes the hermit-rs unikernel target viable: the WIT contract stays stable, and the kernel-side impl swaps from wasmtime-wasi-backed futures to native unikernel primitives.Context: this amendment edits @1.0.0 files
This PR modifies
host/{net,http,process,sys}@1.0.0.witin place rather than shipping@1.1.0variants. One-shot pre-adoption amendment, justified because:mainin feat!: split astrid:capsule@0.1.0 into per-domain frozen packages #7 without any downstream consumer.unicity-astrid/astrid#feat/wit-per-domain-split); no SDK or capsule has bound to the @1.0.0 shapes.The frozen-file CI gate has been retired in commit
8a0f29d— it was tripping on the legitimate pre-adoption amendments and providing no consumer protection (no consumers exist yet). The freeze rule remains documented in the README; re-enable the check from git history once a real downstream consumer ships against a versioned file.Commits
9d71a05fix!: replace wasi:io/poll with astrid:io/poll@1.0.0host/io@1.0.0.witwith thepollableresource andpoll(list<borrow<pollable>>) -> result<list<u32>, error-code>.host/{ipc,net,http,process}@1.0.0.wit:use wasi:io/poll@0.2.0.{pollable}→use astrid:io/poll@1.0.0.{pollable}.host/sys@1.0.0.wit: header rewritten — nowasi:*carve-outs.cfe24c0chore(deps): drop vendored wasi-iodeps/wasi-io/entirely.scripts/validate-wit.sh+.github/workflows/lint.ymlcomments updated.0b66ee2feat(io)!: add astrid:io/error + astrid:io/streamshost/io@1.0.0.witextended with two more interfaces matching thewasi:io@0.2.0shape (error / poll / streams).host/net@1.0.0.wit:tcp-stream.{read-stream, write-stream}— primary throughput primitive for capsule-hosted TCP servers / proxies.host/http@1.0.0.wit:http-stream.body-stream— HTTP response body forwarding.13daba9fix(process)!: drop stream halves on process-handle; mark package desktop-onlyprocess-handlestays byte-oriented (write-stdin/read-logs); no stream accessors.8a0f29dfixup(io): apply Gemini review + drop immutability CI gatepoll()per-call cap raised from 64 → 256 (a capsule at full IPC subscription quota was already past 64).input-stream.read()no longer claims to trap above 1 MiB — wasi-style, only traps on lengths exceeding wasm32's addressable range; the host's 1 MiB ceiling truncates the return instead.output-stream.blocking-write-and-flush()andblocking-write-zeroes-and-flush()no longer cap at 4 KiB. The host segments large transfers internally and yields between chunks.scripts/lint-wit-immutability.sh+ theWIT frozen-file immutabilityworkflow job (see "Context" above).Why Astrid-owned and not wasi:io
Even though
pollable.blockandstreams.readlook like they're "just plumbing," the wasmtime-wasi implementation skips:Owning the namespace lets the kernel wrap every operation in the same gates that cover
astrid:fs,astrid:net, etc. The wasmtime-wasi-io storage types (DynPollable,IoError,DynInputStream,DynOutputStream) are still reused on the desktop kernel — they're futures, not syscalls — but theHosttraits themselves are Astrid's, with audit + cancel + quota wrappers around each call. On hermit-rs, those storage types swap for native unikernel wait/io primitives; the WIT contract is unchanged.Test plan
scripts/validate-wit.shpasses on all 13 host packages locally (wasm-tools 1.245.1).unicity-astrid/astrid#feat/wit-per-domain-split) builds clean against this branch (cargo build --workspace,cargo clippy --workspace -- -D warnings).WIT files parsejob passes.Downstream
unicity-astrid/astrid#feat/wit-per-domain-splitconsumes this branch via the wit submodule. Once this merges, that PR rebases on the new submodule pointer and is ready for review. Kernel-sideMAX_POLL_LIST = 64will be bumped to 256 in a follow-up commit on that branch to match the new WIT.