Repository navigation
gpui_terminal: run GPUI apps inside kitty-graphics terminals - #13
Conversation
Plumbing for hosts that present frames themselves (a terminal): - gpui_apple: MetalRenderer::new_headless is no longer test-only and takes the output pixel format (BGRA8Unorm or RGBA8Unorm). Every pipeline and the path intermediate texture use it, so a consumer that wants RGBA gets it without a CPU swizzle. - gpui_apple: render_scene_to_pixels renders into a reused Shared (Managed on discrete GPUs) texture and copies tightly packed rows into a caller slice, e.g. a mmap'd shared memory object. - gpui_macos: export MetalRenderer, InstanceBufferPool and MacTextSystem. - gpui: gpui::queue (PriorityQueueSender/Receiver) is no longer gated off macOS, so a custom macOS dispatcher can use it. Discord: https://discord.com/channels/1422625037164351591/1562047194893131797 Session: ses_ea72eb238ffeNBQNOGK7liwLQG
New GPUI platform (macOS for now). The window is the terminal grid. Frames: Metal offscreen render in RGBA, read straight into a POSIX shm object, shown with ESC_G a=T,i=1,p=1,f=32,t=s inside CSI ?2026 sync, so the pty only carries ~100 bytes per frame. Reusing image/placement id replaces the frame in place without flicker. Frames are demand driven (frame_waker/schedule_frame) and capped at 60 Hz. Each frame runs in an autorelease pool because there is no Cocoa run loop. Input: /dev/tty in raw mode, SGR any-motion mouse with pixel coordinates (1003/1006/1016), kitty keyboard flags 31 (press/repeat/release, alternate keys, associated text), bracketed paste, focus reports, in-band resize (2048) plus SIGWINCH. The input thread uses select() because macOS poll() reports POLLNVAL for /dev/tty. Text goes through the input handler when a key is unhandled; unhandled ctrl-c quits. Title via OSC 2, pointer shape via OSC 22, clipboard write via OSC 52. Terminal is restored on quit, signals and panic. Main loop: one mpsc channel carries GPUI main-thread runnables and terminal events; background work uses gpui's priority queue; timers run on their own thread. Verified in Ghostty 1.3.1 (hover, clicks, wheel, keys, resize) and with a fake kitty pty harness that reads shm frames into PNGs. ~4 ms per 1600x960 frame on Apple silicon. Example: cargo run -p gpui_terminal --example terminal_hello Discord: https://discord.com/channels/1422625037164351591/1562047194893131797 Session: ses_ea72eb238ffeNBQNOGK7liwLQG
gpui_wgpu: - WgpuContext::new_offscreen picks an adapter without a window surface (Backends::PRIMARY | GL, software adapters accepted, so it works on machines without a GPU via lavapipe/llvmpipe). - WgpuResources.surface is Option; None means offscreen. New WgpuRenderer::new_offscreen and render_to_pixels: render into a reused RGBA texture, copy to a reused MAP_READ buffer (rows padded to 256 bytes), wait for the submission, unpad rows into the caller slice. draw() returns false without a surface. Globals upload is factored out of draw() into write_globals(). Windowed paths are unchanged. gpui_terminal: - Builds on macOS, Linux and FreeBSD. Linux uses CosmicTextSystem and the wgpu offscreen renderer; macOS keeps Metal + CoreText. - open(1) on macOS, xdg-open(1) elsewhere. OSC 52 primary selection for write_to_primary. - Linux shm pages are reserved with posix_fallocate, so a full /dev/shm fails with ENOSPC instead of a SIGBUS during the readback write. Verified in arm64 Debian Docker with mesa lavapipe: unit tests, clippy, and a fake kitty pty harness (hover, clicks, scroll, keys, resize, ctrl-c, odd sizes that exercise row padding). Discord: https://discord.com/channels/1422625037164351591/1562047194893131797 Session: ses_ea72eb238ffeNBQNOGK7liwLQG
GPUIX runs GPUI on Node's main thread and pumps it from JS. The terminal platform now supports that shape, like MacPlatform::new_embedded: - TerminalPlatform::new_embedded(): Platform::run returns after launching. - pump_events(): runs queued main-thread work (8 ms budget, then re-wakes) and a due frame without blocking. Returns false once the app quit; the terminal is restored by then. - set_wake_hook(): fires from any thread when the main thread has work. MainSender wakes after every runnable and terminal event, the window wakes when a frame is first requested, and a throttled frame schedules a wake on the timer thread at its deadline. Also, for any host: - Signal handlers use sigaction and forward to the handlers installed before them (Node's libuv), instead of replacing them. SIG_DFL is not forwarded, so SIGTERM/SIGINT quit and restore the terminal. - atexit restores the terminal, so exit(3) without unwinding (process.exit) does not leave raw mode and the alternate screen. - stdout/stderr that are the terminal are redirected to an unlinked temp file while the app runs, then replayed after restore. A print would otherwise scroll the alternate screen and the frame with it. Discord: https://discord.com/channels/1422625037164351591/1562055543466496043 Session: ses_ea7105a47ffeNLpB02sEyo5t9g
Bun installs its own SIGTERM handler, which exits on the spot. Forwarding to it killed the process before the terminal was restored. Quit signals now only quit the app; the host exits after the terminal is restored. SIGWINCH is still forwarded so Node keeps seeing resizes. Discord: https://discord.com/channels/1422625037164351591/1562055543466496043 Session: ses_ea7105a47ffeNLpB02sEyo5t9g
Over SSH the terminal runs on another machine and cannot open our POSIX shared memory, so every `t=s` frame failed silently and the screen stayed empty. Detection, like kitty's icat, yazi and ratatui-image: - at window open, send a one pixel `a=q,t=s,f=24,S=3` query on a 3 byte shm object, then `CSI c` - OK → shared memory, unchanged path - error, a DA1 reply first (replies come in order), or no answer in 2 s → inline Inline frames: full RGBA frame, zlib fast (flate2), base64, 4096 byte chunks (`t=d,o=z`, `m=1`..`m=0`, `q=2`) inside the 2026 brackets. Each frame is followed by a one pixel `a=q` fence. No new frame renders while two fences wait for a reply, so SSH and TCP buffers cannot queue seconds of stale frames. A fence without a reply for 2 s stops counting. Parser: `ESC_G i=..;OK|error` becomes TerminalEvent::GraphicsReply and `CSI ? .. c` becomes TerminalEvent::DeviceAttributes. | | bytes per 1600x960 frame | |---|---| | shared memory | ~100 | | inline | ~120 KB (terminal_hello) | Tested with the fake kitty harness in local mode (shm chosen), in remote mode (inline), and with the app in Docker and the harness on the host, where /dev/shm really differs. Writes stay on the main thread; a slow link blocks it while it drains. Discord: https://discord.com/channels/1422625037164351591/1562061946524532766 Session: ses_ea6f90cc0ffej0S2RFLCkQxS5W
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8152a6a875
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for signal in [libc::SIGWINCH, libc::SIGTERM, libc::SIGHUP, libc::SIGINT] { | ||
| let mut previous = unsafe { std::mem::zeroed::<libc::sigaction>() }; | ||
| unsafe { libc::sigaction(signal, &action, &mut previous) }; | ||
| if signal == libc::SIGWINCH { | ||
| PREVIOUS_RESIZE_ACTION.get_or_init(|| previous); | ||
| } |
There was a problem hiding this comment.
In embedded hosts, these handlers remain installed after TerminalPlatform::shut_down() only restores the TTY. Consequently, after the terminal application quits, subsequent SIGINT, SIGTERM, or SIGHUP signals are still intercepted and written to the signal pipe instead of invoking the host's previous handler or default action; a continuing Node host can therefore stop responding to Ctrl-C or termination requests. Preserve every previous action and restore it while closing the signal pipe during shutdown.
Useful? React with 👍 / 👎.
| display: Rc::new(TerminalDisplay { | ||
| bounds: Bounds::new( | ||
| Point::default(), | ||
| size( | ||
| px(terminal_size.width as f32), | ||
| px(terminal_size.height as f32), | ||
| ), |
There was a problem hiding this comment.
When TIOCGWINSZ reports ws_xpixel/ws_ypixel as zero, which is common for PTYs, the platform permanently exposes a 0×0 primary display even though TerminalWindow::new later applies the 8×16-cell fallback and renders a nonzero window. On inferred HiDPI terminals these values are also device pixels rather than the logical pixels returned by PlatformWindow::bounds(). APIs such as cx.primary_display() and Bounds::maximized therefore receive bounds inconsistent with the actual terminal window; apply the same fallback and scale conversion here, and keep the display bounds synchronized on resize.
Useful? React with 👍 / 👎.
| let target = match self.transport { | ||
| Transport::Probing { .. } => anyhow::bail!("the frame transport is not known yet"), | ||
| Transport::SharedMemory => Target::Shared(self.create_shared_memory(length)?), | ||
| Transport::Inline => { |
There was a problem hiding this comment.
After the tiny startup probe selects shared memory, each full-size frame allocation can still fail—for example when /dev/shm has room for the probe but not a large frame, or while an earlier frame still occupies the available space. This ? aborts drawing, and WindowInner::draw only logs the error after run_frame_if_due has cleared frame_requested, so a static scene can remain blank or stale indefinitely. Switch the presenter to inline transport (or schedule a retry) when a per-frame shared-memory allocation fails.
Useful? React with 👍 / 👎.
New GPUI platform crate
gpui_terminal: the window is the terminal grid, frames render offscreen (Metal on macOS, wgpu on Linux) and show with the kitty graphics protocol. GPUIX uses it forterminal: true.What's in it
gpui: offscreen Metal rendering for non-window hostsgpui_terminal: platform, input parser (SGR pixel mouse, kitty keyboard, paste, focus, resize), Linux via wgpu, embedded mode for Node, SIGWINCH-only forwardinga=q,t=sprobe +CSI cat startup. No shared memory → frames go inline (zlib, base64, chunked) with ana=qfence, max 2 frames in flightTested in Ghostty and with a fake kitty harness, also with the app in Docker and the harness on the host (real separate
/dev/shm).