Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ plus closed models via fal.ai.
> `cd core && python main.py --front-end-root ../dist-web` runs Core and serves the built UI on one
> port.

> **GitHub org: `inlineresearch`.** Every repo lives there - never `inline-studio/` or any other
> org in a URL, manifest, or doc.
>
> | Repo | What it is |
> | ----------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
> | [`inlineresearch/Inline-Studio`](https://github.com/inlineresearch/Inline-Studio) | This repo: the UI client + Inline Core |
> | [`inlineresearch/Inline-Core`](https://github.com/inlineresearch/Inline-Core) | The engine's own repo (subtree source for `core/`) |
> | [`inlineresearch/Inline-Registry`](https://github.com/inlineresearch/Inline-Registry) | The published extension index the Available tab reads |
> | [`inlineresearch/Inline-Studio-Extension-Guide`](https://github.com/inlineresearch/Inline-Studio-Extension-Guide) | The reference extension authors copy |

> Read this file before changing code. It defines the architecture and the non-negotiable rules.

## Mental model (everything is organised around this)
Expand Down Expand Up @@ -121,6 +131,10 @@ clear "not available" for them.
(`src/shared/nodes/`); their execution is Core's fal relay.
- **Files & naming.** Components `PascalCase.tsx`, hooks `useX.ts`, one component per file,
feature-foldered views. Keep files under ~300 lines without a good reason.
- **Comments are short.** One or two lines, and only for the **why** a reader can't infer from the
code - a non-obvious constraint, a rejected alternative, an ordering that matters. Never narrate
what the code does or restate the line below. If the reasoning needs paragraphs, it belongs in a
doc, not in the source.
- **Icons, never emoji.** Never use emoji in the UI (no 🎬/🎵/✂/🔊 as glyphs). Use crisp,
consistent line SVG icons (Lucide-style: `viewBox="0 0 24 24"`, `fill="none"`,
`stroke="currentColor"`) that inherit color/size via `currentColor` + a size class. Follow the
Expand Down
31 changes: 30 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ It runs as a **single process on one port**: the Inline Core engine (Python) ser
- **Video editing on the canvas** - the **Video Director node** is a timeline-in-a-node that assembles your rendered frames into a single cut, with layered audio (the videos' own audio plus your own music/VO), per-input and per-layer volume, an in-node preview to scrub, and high-res export; the **Trim Video/Audio node** lets you drop in a clip, drag the in/out handles over its filmstrip/waveform, and pass just the trimmed segment downstream.
- **Local generation, built in** - the Inline Core engine runs diffusion models on your own GPU. Z-Image Turbo from a single model file, no external server to set up.
- **API Nodes for hosted models** - run closed models right on the canvas with no GPU. Add a Generate node, pick a model, and bring your own provider key. See [API Nodes](#api-nodes).
- **Community extensions** - install custom nodes from a GitHub repo in one click, security-reviewed and dependency-isolated. Browse the [registry](https://github.com/inlineresearch/Inline-Registry) or [build your own](https://github.com/inlineresearch/Inline-Studio-Extension-Guide).
- **Free & open source (MIT)** - one process (Python + a browser); runs on macOS, Windows, and Linux.

[**Follow our Animated Short Film with LTX 2.3 and GPT Image Generation tutorial →**](https://inlinestudio.art/projects/circuit-race)
Expand Down Expand Up @@ -75,12 +76,30 @@ Inline Core is a from-scratch generation engine for local rendering. It keeps th
- **A single device policy owns all placement** - device, dtype, offload, and attention, so the same graph runs on a 4090, a 6 GB laptop, pure CPU, or split across several GPUs without touching the graph.
- **Bring your own models, no hidden downloads** - a drop-in `models/` layout feeds a typed catalog and versioned node descriptors; nothing is fetched behind your back.

### Multi-GPU: split one image across GPUs
### Community extensions

Install community-built nodes straight from a GitHub repo, from the Extensions dialog or a repo URL.

- **One-click install**, with a live stepper showing download, security review, dependency resolution, and activation.
- **Every install is reviewed.** Code that could replace Inline's PyTorch, hide a payload, or run at install time is blocked outright; subprocesses, sockets, and unknown network hosts need your explicit approval.
- **Extensions can't break your setup.** Their dependencies install into their own folder and can never touch the shared torch/diffusers runtime, and genuine conflicts fail at install with both versions named.
- **Nodes appear on the canvas immediately**, with their own params, model downloads, take history, and Run control. No restart, and no frontend code from the author.
- **Toggle any node on or off**, roll back to a previous version, or uninstall, and see when an update is available.
- **Publish by tagging.** Authors list once in the registry; after that a new tag reaches users with no further PR.

Browse the [**extension registry**](https://github.com/inlineresearch/Inline-Registry), or copy the
[**extension guide**](https://github.com/inlineresearch/Inline-Studio-Extension-Guide) to build your
own: four working nodes, declared model downloads, and a full authoring reference.

<details>
<summary><b>Multi-GPU: split one image across GPUs</b></summary>

Got two or more GPUs? Inline Core can cut a single image's latency by running its **denoise loop** (the expensive, iterative sampling step) collectively across them. This is not "one image per GPU" (independent renders); it's **one image whose sampling is shared by all the GPUs**, so a single render finishes faster.

It's done with [xDiT](https://github.com/xdit-project/xDiT) (`xfuser`), which parallelizes diffusion-transformer inference in an isolated worker group (one process per GPU via `torchrun`, over local IPC). The HTTP server, database, and graph stay single-process; only the denoise distributes, and it sits behind a sampler seam so single-GPU/CPU runs pay no overhead. The split method is chosen from the interconnect Core detects: **PipeFusion** (default, works over plain PCIe) or **Ulysses** (sequence-parallel attention, used when NVLink is present). Turn it on with `./webui.sh --multi-gpu` after `uv pip install -e ".[parallel]"`.

</details>

For the full engineering story (the graph/sampler/device-policy design, the node vocabularies, and the xDiT worker group), see **[core/README.md](core/README.md)** and **[core/CLAUDE.md](core/CLAUDE.md)**.

## API Nodes
Expand Down Expand Up @@ -114,6 +133,9 @@ Prefer pip? `pip install -r requirements.txt` (from the repo root) pulls the eng

### Hardware support

<details>
<summary><b>GPU, CPU, Apple Silicon, and ROCm setup</b></summary>

Honest status - what's actually been run, versus what has a code path but no one has verified:

| Hardware | Status | Extra steps |
Expand Down Expand Up @@ -153,8 +175,13 @@ Two gotchas:
- **Local model coverage is Z-Image Turbo only** today. Flux, SDXL and others are planned; hosted models via [API Nodes](#api-nodes) need no GPU at all.
- **1024² with Guidance (CFG) above 0 needs more than 16 GB.** CFG runs the prompt and negative prompt together, doubling the denoise. Z-Image Turbo is distilled to run CFG-free - at Guidance 0, 1024² fits in ~11.5 GB.

</details>

### From source (for UI development)

<details>
<summary><b>Build the UI and run the engine locally</b></summary>

To hack on the web UI you need [Node.js](https://nodejs.org) 20.11+ as well, and you serve a local SPA build:

```bash
Expand All @@ -174,6 +201,8 @@ Then open **http://127.0.0.1:8848**. Add your [fal.ai API key](https://fal.ai/da

**Hot-reload:** run the engine as above, then in another terminal `npm run dev:web` (Vite serves the UI with HMR and proxies API calls to Core).

</details>

### Command-line options

The friendly `webui.sh` launcher (in `core/`) maps flags onto the engine's `INLINE_*` environment knobs; `core/main.py` takes the same flags when you run the engine directly. `./webui.sh --help` lists them all.
Expand Down
17 changes: 14 additions & 3 deletions core/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@ models across macOS, Windows, and Linux - from CPU-only boxes and low-VRAM lapto
machines that split a single image's sampling across GPUs (via xDiT). **It is the render backend that
replaces ComfyUI for Inline.**

> The UI client lives in the separate **Inline Studio / Storyline** repo (`inline-studio`, an Electron
> app). It drives this engine over the `/v1` HTTP + websocket API. Inline Core is headless and knows
> nothing about the UI.
> The UI client lives in the separate **Inline Studio** repo
> ([`inlineresearch/Inline-Studio`](https://github.com/inlineresearch/Inline-Studio)), which vendors
> this engine under `core/` via `git subtree`. It drives the engine over the `/v1` HTTP + websocket
> API; Inline Core is headless and knows nothing about the UI.

> **GitHub org: `inlineresearch`** - never `inline-studio/` or any other org in a URL, manifest, or
> doc. Sibling repos: `Inline-Studio` (UI + this engine), `Inline-Registry` (the published extension
> index served to the Available tab), `Inline-Studio-Extension-Guide` (the reference extension).

> Read this file before changing code. It defines the architecture and the non-negotiable rules.
> `README.md` is the user/product-facing version of the same story; this is the engineering contract.
Expand Down Expand Up @@ -170,6 +175,12 @@ real codec that moves tensors lives with the model runner.

- **Typed, strict.** `pyright` in strict mode (`[tool.pyright]`, `typeCheckingMode = "strict"`), all of
`src` + `tests`. No silent `Any` leaks across component/graph boundaries.
- **Comments are short.** One or two lines, and only for the **why** a reader can't infer from the
code - a non-obvious constraint, a rejected alternative, an ordering that matters. Module
docstrings: 1-3 sentences. Function docstrings: one line, or none when the signature says it.
Never narrate what the code does, never write an essay in a docstring, never leave a comment that
restates the line below it. If the reasoning genuinely needs paragraphs, it belongs in a doc, not
in the source.
- **Lint.** `ruff` with `select = ["E", "F", "I", "UP", "B"]`, line length 100, target `py311`.
- **Typed graph, validated before run.** Never execute an unvalidated graph. Edge type-checking
(`graph/validate.py` + `port_satisfies`) rejects bad wiring at submit. New port kinds go in
Expand Down
14 changes: 14 additions & 0 deletions core/extensions/.cache/registry.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"entries": [
{
"id": "demo",
"name": "Inline Demo Extension",
"description": "A reference extension: a runnable generate node plus three image nodes.",
"repo": "https://github.com/inlineresearch/Inline-Extension-Reference",
"homepage": "https://github.com/inlineresearch/Inline-Extension-Reference",
"author": "Inline Research",
"tags": ["reference", "image"]
}
],
"etag": "\"bdbc10b5cdc9f8646693623ba904dda4898bd71f583125fe2eeb69da1968a01e\""
}
17 changes: 17 additions & 0 deletions core/src/inline_core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,23 @@ def data_dir() -> Path:
return Path(env).expanduser() if env else Path(".inline")


def extensions_dir() -> Path:
"""Community extensions root. `INLINE_EXTENSIONS_DIR`, else `./extensions` (so a dev checkout
keeps it beside `./models` and `./.inline`). Holds `state.json`, the host constraint snapshot,
and one directory per installed extension."""
env = os.environ.get("INLINE_EXTENSIONS_DIR")
return Path(env).expanduser() if env else Path("extensions")


def registry_url() -> str:
"""Where the Available tab fetches its extension index. `INLINE_EXTENSION_REGISTRY`, else the
public registry. Point it at a fork or a file:// path to test a registry change."""
return os.environ.get(
"INLINE_EXTENSION_REGISTRY",
"https://raw.githubusercontent.com/inlineresearch/Inline-Registry/main/index.json",
)


def server_host() -> str:
"""Address the /v1 server binds. `INLINE_HOST`, else loopback (`127.0.0.1`)."""
return os.environ.get("INLINE_HOST", "127.0.0.1")
Expand Down
10 changes: 10 additions & 0 deletions core/src/inline_core/extensions/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
"""Community extensions: installable node extensions from a git repo.

A **extension** is one repo, installed at one pinned commit, holding N independently toggleable
**modules**, each owning node types, optional model requirements, and optional UI.

Authors import only ``inline_core.extensions.api``.

Dependencies get private resolution with install-time conflict detection - not runtime isolation.
See ``importer`` for why the distinction matters.
"""
Loading