Skip to content

std.ipc: two-way socket channel with descriptor passing #3490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
paul-hammant merged 4 commits into from
Oct 31, 2026
Merged

std.ipc: two-way socket channel with descriptor passing #3490

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4998d4e
feat(ipc): support socket channel and fd passing
google-labs-jules[bot] Oct 31, 2026
cfbde83
Merge branch 'main' into feat/ipc-socket-fd-passing
paul-hammant Oct 31, 2026
56ff4ca
std.ipc: shape the socket channel and descriptor passing
paul-hammant Oct 31, 2026
a3d51be
std.ipc: recv_fd sees every descriptor; spawn builds without sockets
paul-hammant Oct 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -960,6 +960,7 @@ TEST_SRC = tests/runtime/test_harness.c \
tests/runtime/test_runtime_observe.c \
tests/runtime/test_runtime_process_mem.c \
tests/runtime/test_spawn_env.c \
tests/runtime/test_ipc_write.c \
tests/runtime/test_runtime_schema.c \
tests/runtime/test_runtime_intmap.c \
tests/runtime/test_tools_sha256.c tools/ae_sha256.c \
Expand Down
1 change: 1 addition & 0 deletions compiler/aetherc.c
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 16 additions & 3 deletions docs/api/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,8 +275,8 @@ <h3>io</h3>
</a>
<a href="ipc.html" class="module-card">
<h3>ipc</h3>
<p>std.ipc — child-to-parent back-channel IPC.</p>
<span class="count">4 exports</span>
<p>std.ipc — a process's channel to its parent: a report pipe, or a two-way socket that can also pass open file descriptors.</p>
<span class="count">13 exports</span>
</a>
<a href="json.html" class="module-card">
<h3>json</h3>
Expand Down Expand Up @@ -376,7 +376,7 @@ <h3>observe</h3>
<a href="os.html" class="module-card">
<h3>os</h3>
<p>std.os - Shell &amp; Process Execution Import with: import std.os</p>
<span class="count">80 exports</span>
<span class="count">84 exports</span>
</a>
<a href="path.html" class="module-card">
<h3>path</h3>
Expand Down Expand Up @@ -1517,9 +1517,18 @@ <h2>Search Results</h2>
<li class="function-item" data-name="fd_read_line"><a href="io.html#fd_read_line"><span class="fn-name">fd_read_line</span><span class="fn-module">io</span></a></li>
<li class="function-item" data-name="fd_read_line_status"><a href="io.html#fd_read_line_status"><span class="fn-name">fd_read_line_status</span><span class="fn-module">io</span></a></li>
<li class="function-item" data-name="parent_channel_raw"><a href="ipc.html#parent_channel_raw"><span class="fn-name">parent_channel_raw</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="parent_channel_kind_raw"><a href="ipc.html#parent_channel_kind_raw"><span class="fn-name">parent_channel_kind_raw</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="write_raw"><a href="ipc.html#write_raw"><span class="fn-name">write_raw</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="read_raw"><a href="ipc.html#read_raw"><span class="fn-name">read_raw</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="send_fd_raw"><a href="ipc.html#send_fd_raw"><span class="fn-name">send_fd_raw</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="recv_fd_raw"><a href="ipc.html#recv_fd_raw"><span class="fn-name">recv_fd_raw</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="parent_channel"><a href="ipc.html#parent_channel"><span class="fn-name">parent_channel</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="parent_channel_kind"><a href="ipc.html#parent_channel_kind"><span class="fn-name">parent_channel_kind</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="write"><a href="ipc.html#write"><span class="fn-name">write</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="write_close"><a href="ipc.html#write_close"><span class="fn-name">write_close</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="read"><a href="ipc.html#read"><span class="fn-name">read</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="send_fd"><a href="ipc.html#send_fd"><span class="fn-name">send_fd</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="recv_fd"><a href="ipc.html#recv_fd"><span class="fn-name">recv_fd</span><span class="fn-module">ipc</span></a></li>
<li class="function-item" data-name="parse_raw"><a href="json.html#parse_raw"><span class="fn-name">parse_raw</span><span class="fn-module">json</span></a></li>
<li class="function-item" data-name="last_error"><a href="json.html#last_error"><span class="fn-name">last_error</span><span class="fn-module">json</span></a></li>
<li class="function-item" data-name="stringify_raw"><a href="json.html#stringify_raw"><span class="fn-name">stringify_raw</span><span class="fn-module">json</span></a></li>
Expand Down Expand Up @@ -2134,6 +2143,10 @@ <h2>Search Results</h2>
<li class="function-item" data-name="run_pipe"><a href="os.html#run_pipe"><span class="fn-name">run_pipe</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="wait_pid"><a href="os.html#wait_pid"><span class="fn-name">wait_pid</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="run_pipe_drain_and_wait"><a href="os.html#run_pipe_drain_and_wait"><span class="fn-name">run_pipe_drain_and_wait</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="run_socket_raw"><a href="os.html#run_socket_raw"><span class="fn-name">run_socket_raw</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="socketpair_raw"><a href="os.html#socketpair_raw"><span class="fn-name">socketpair_raw</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="run_socket"><a href="os.html#run_socket"><span class="fn-name">run_socket</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="socketpair"><a href="os.html#socketpair"><span class="fn-name">socketpair</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="spawn_proc"><a href="os.html#spawn_proc"><span class="fn-name">spawn_proc</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="wait"><a href="os.html#wait"><span class="fn-name">wait</span><span class="fn-module">os</span></a></li>
<li class="function-item" data-name="wait_any"><a href="os.html#wait_any"><span class="fn-name">wait_any</span><span class="fn-module">os</span></a></li>
Expand Down
62 changes: 59 additions & 3 deletions docs/api/ipc.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,18 @@ <h1><a href="index.html">Aether</a></h1>
<h2>Exports</h2>
<ul class="function-nav">
<li><a href="#parent_channel_raw">parent_channel_raw</a></li>
<li><a href="#parent_channel_kind_raw">parent_channel_kind_raw</a></li>
<li><a href="#write_raw">write_raw</a></li>
<li><a href="#read_raw">read_raw</a></li>
<li><a href="#send_fd_raw">send_fd_raw</a></li>
<li><a href="#recv_fd_raw">recv_fd_raw</a></li>
<li><a href="#parent_channel">parent_channel</a></li>
<li><a href="#parent_channel_kind">parent_channel_kind</a></li>
<li><a href="#write">write</a></li>
<li><a href="#write_close">write_close</a></li>
<li><a href="#read">read</a></li>
<li><a href="#send_fd">send_fd</a></li>
<li><a href="#recv_fd">recv_fd</a></li>
</ul>
<hr>
<h2>Modules</h2>
Expand Down Expand Up @@ -118,32 +127,79 @@ <h2>Modules</h2>
<main class="content">
<div class="module-header">
<h1>ipc</h1>
<p class="module-desc">std.ipc — child-to-parent back-channel IPC.</p>
<p class="module-desc">std.ipc — a process's channel to its parent: a report pipe, or a two-way socket that can also pass open file descriptors.</p>
<p class="import-line"><code>import std.ipc</code></p>
</div>
<div class="functions" id="functions">
<div class="function" id="parent_channel_raw" data-name="parent_channel_raw">
<h3 class="function-name">parent_channel_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>parent_channel_raw() <span class="arrow">-&gt;</span> int</code></pre>
<p class="doc">Returns the parent-channel fd if the parent spawned this process via os.run_pipe (i.e. AETHER_IPC_FD is set in env and the named fd is open); -1 otherwise. Children of os.run / os.run_capture see -1 — those don't open a back-channel. On Windows always -1 (v1 stub). The fd value is whatever the parent picked; current parent-side default is 3 but consumers MUST NOT hardcode that.</p>
<p class="doc">Returns the parent-channel fd if the parent spawned this process via os.run_pipe or os.run_socket (i.e. AETHER_IPC_FD is set in env and the named fd is open); -1 otherwise. Children of os.run / os.run_capture see -1 — those don't open a back-channel. On Windows always -1 (v1 stub). The fd value is whatever the parent picked; current parent-side default is 3 but consumers MUST NOT hardcode that.</p>
</div>
<div class="function" id="parent_channel_kind_raw" data-name="parent_channel_kind_raw">
<h3 class="function-name">parent_channel_kind_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>parent_channel_kind_raw() <span class="arrow">-&gt;</span> string</code></pre>
</div>
<div class="function" id="write_raw" data-name="write_raw">
<h3 class="function-name">write_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>write_raw(fd, bytes) <span class="arrow">-&gt;</span> string</code></pre>
<p class="signature"><span class="sig-params">fd: int, bytes: string</span></p>
</div>
<div class="function" id="read_raw" data-name="read_raw">
<h3 class="function-name">read_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>read_raw(fd, max) <span class="arrow">-&gt;</span> (string, string)</code></pre>
<p class="signature"><span class="sig-params">fd: int, max: int</span></p>
</div>
<div class="function" id="send_fd_raw" data-name="send_fd_raw">
<h3 class="function-name">send_fd_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>send_fd_raw(ch, fd, payload) <span class="arrow">-&gt;</span> string</code></pre>
<p class="signature"><span class="sig-params">ch: int, fd: int, payload: string</span></p>
</div>
<div class="function" id="recv_fd_raw" data-name="recv_fd_raw">
<h3 class="function-name">recv_fd_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>recv_fd_raw(ch, max) <span class="arrow">-&gt;</span> (int, string, string)</code></pre>
<p class="signature"><span class="sig-params">ch: int, max: int</span></p>
</div>
<div class="function" id="parent_channel" data-name="parent_channel">
<h3 class="function-name">parent_channel <span class="kind">fn</span></h3>
<pre class="usage"><code>parent_channel() <span class="arrow">-&gt;</span> int</code></pre>
<p class="doc">Aether-side wrappers. parent_channel returns the raw int from the extern; -1 sentinel for &quot;no parent channel.&quot;</p>
</div>
<div class="function" id="parent_channel_kind" data-name="parent_channel_kind">
<h3 class="function-name">parent_channel_kind <span class="kind">fn</span></h3>
<pre class="usage"><code>parent_channel_kind() <span class="arrow">-&gt;</span> string</code></pre>
<p class="doc">&quot;socket&quot; for a channel from os.run_socket (two-way, can pass descriptors), &quot;pipe&quot; for one from os.run_pipe (write-only from the child), &quot;&quot; when parent_channel() finds none.</p>
</div>
<div class="function" id="write" data-name="write">
<h3 class="function-name">write <span class="kind">fn</span></h3>
<pre class="usage"><code>write(fd, bytes) <span class="arrow">-&gt;</span> string</code></pre>
<p class="signature"><span class="sig-params">fd: int, bytes: string</span></p>
<p class="doc">Write `bytes` to the back-channel `fd`. Returns &quot;&quot; on success, non-empty error string on failure. Wraps std.net.fd_write — the underlying transport is an inherited pipe, so the same byte-stream semantics apply (partial writes are possible on interrupted system calls; the wrapped C extern handles EINTR). For the common case where the child writes one payload and exits, prefer write_close so the parent's read_all loop sees EOF promptly.</p>
<p class="doc">Write all of `bytes` to `fd`. Returns &quot;&quot; on success, non-empty error string on failure. A short write continues where it stopped and an interrupted one retries, so &quot;&quot; means every byte went; a string with a zero byte in it goes whole. On a socket whose peer has gone this is an error, not SIGPIPE. For the common case where the child writes one payload and exits, prefer write_close so the parent's read_all loop sees EOF promptly.</p>
</div>
<div class="function" id="write_close" data-name="write_close">
<h3 class="function-name">write_close <span class="kind">fn</span></h3>
<pre class="usage"><code>write_close(fd, bytes) <span class="arrow">-&gt;</span> string</code></pre>
<p class="signature"><span class="sig-params">fd: int, bytes: string</span></p>
<p class="doc">Write `bytes` then close the channel. The common case: child produces one structured report at exit and closes the fd so the parent's drain loop sees EOF and stops. Calling close releases the writer's reference on the pipe; the parent sees EOF only after every writer has closed. If the child is itself going to exit immediately after, the kernel will close the fd anyway during process teardown — explicit close is still preferred because it lets the parent's reader unblock before the child exits, useful when the parent's reader runs in the same thread as the child-wait loop.</p>
</div>
<div class="function" id="read" data-name="read">
<h3 class="function-name">read <span class="kind">fn</span></h3>
<pre class="usage"><code>read(ch, max) <span class="arrow">-&gt;</span> (string, string)</code></pre>
<p class="signature"><span class="sig-params">ch: int, max: int</span></p>
<p class="doc">One read of up to `max` bytes: (data, &quot;&quot;) with what arrived, which may be less than was written (the channel is a byte stream; framing is the caller's), (&quot;&quot;, &quot;&quot;) at end of stream, or (&quot;&quot;, err). Binary-safe. Blocks until something arrives.</p>
</div>
<div class="function" id="send_fd" data-name="send_fd">
<h3 class="function-name">send_fd <span class="kind">fn</span></h3>
<pre class="usage"><code>send_fd(ch, fd, payload) <span class="arrow">-&gt;</span> string</code></pre>
<p class="signature"><span class="sig-params">ch: int, fd: int, payload: string</span></p>
<p class="doc">Pass the open descriptor `fd` to the process at the other end of socket `ch`, attached to `payload`, which must be at least one byte (on a stream socket a descriptor travels with bytes). The receiver gets its own descriptor for the same open file; this side's `fd` stays open, so close it when this side is done with it.</p>
</div>
<div class="function" id="recv_fd" data-name="recv_fd">
<h3 class="function-name">recv_fd <span class="kind">fn</span></h3>
<pre class="usage"><code>recv_fd(ch, max) <span class="arrow">-&gt;</span> (int, string, string)</code></pre>
<p class="signature"><span class="sig-params">ch: int, max: int</span></p>
<p class="doc">Receive one descriptor from socket `ch`, with the bytes that came with it (up to `max`): (fd, payload, &quot;&quot;), or (-1, &quot;&quot;, err). The descriptor is close-on-exec. A message carrying no descriptor, more than one, or control data of another kind is an error, and every descriptor that did arrive is closed first.</p>
</div>
</div>
</main>
<script src="search.js"></script>
Expand Down
24 changes: 24 additions & 0 deletions docs/api/os.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,10 @@ <h2>Exports</h2>
<li><a href="#run_pipe">run_pipe</a></li>
<li><a href="#wait_pid">wait_pid</a></li>
<li><a href="#run_pipe_drain_and_wait">run_pipe_drain_and_wait</a></li>
<li><a href="#run_socket_raw">run_socket_raw</a></li>
<li><a href="#socketpair_raw">socketpair_raw</a></li>
<li><a href="#run_socket">run_socket</a></li>
<li><a href="#socketpair">socketpair</a></li>
<li><a href="#spawn_proc">spawn_proc</a></li>
<li><a href="#wait">wait</a></li>
<li><a href="#wait_any">wait_any</a></li>
Expand Down Expand Up @@ -506,6 +510,26 @@ <h3 class="function-name">run_pipe_drain_and_wait <span class="kind">fn</span></
<p class="signature"><span class="sig-params">prog: string, argv: ptr, env: ptr</span></p>
<p class="doc">Convenience: spawn + drain back-channel pipe + wait + reap, all in one call. Returns (channel_payload, exit_code, err). Recommended entry point for &quot;child writes structured report at exit; parent reads it after.&quot; No pipe-buffer deadlock risk regardless of payload size. On Windows, returns (&quot;&quot;, -1, &quot;unsupported on Windows&quot;).</p>
</div>
<div class="function" id="run_socket_raw" data-name="run_socket_raw">
<h3 class="function-name">run_socket_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>run_socket_raw(prog, argv, env) <span class="arrow">-&gt;</span> (int, int, string)</code></pre>
<p class="signature"><span class="sig-params">prog: string, argv: ptr, env: ptr</span></p>
</div>
<div class="function" id="socketpair_raw" data-name="socketpair_raw">
<h3 class="function-name">socketpair_raw <span class="kind">extern</span></h3>
<pre class="usage"><code>socketpair_raw() <span class="arrow">-&gt;</span> (int, int, string)</code></pre>
</div>
<div class="function" id="run_socket" data-name="run_socket">
<h3 class="function-name">run_socket <span class="kind">fn</span></h3>
<pre class="usage"><code>run_socket(prog, argv, env) <span class="arrow">-&gt;</span> (int, int, string)</code></pre>
<p class="signature"><span class="sig-params">prog: string, argv: ptr, env: ptr</span></p>
<p class="doc">Spawn child with std.ipc socket channel (AF_UNIX socketpair). Returns (parent_socket_fd, child_pid, err). Child sees AETHER_IPC_FD=3 and AETHER_IPC_KIND=socket in its environment; std.ipc.parent_channel() resolves to fd 3 and std.ipc.parent_channel_kind() resolves to &quot;socket&quot;. Parent end is close-on-exec. Caller MUST call os.wait_pid(child_pid) to reap. On Windows / non-filesystem targets, returns (-1, -1, &quot;unsupported on Windows&quot; or error).</p>
</div>
<div class="function" id="socketpair" data-name="socketpair">
<h3 class="function-name">socketpair <span class="kind">fn</span></h3>
<pre class="usage"><code>socketpair() <span class="arrow">-&gt;</span> (int, int, string)</code></pre>
<p class="doc">Create a connected pair of Unix domain sockets (AF_UNIX, SOCK_STREAM). Returns (fd_a, fd_b, err). Both ends are close-on-exec. On Windows / non-filesystem targets, returns (-1, -1, &quot;unsupported on Windows&quot; or error).</p>
</div>
<div class="function" id="spawn_proc" data-name="spawn_proc">
<h3 class="function-name">spawn_proc <span class="kind">fn</span></h3>
<pre class="usage"><code>spawn_proc(prog, argv, env) <span class="arrow">-&gt;</span> (int, string)</code></pre>
Expand Down
Loading
Loading