Skip to content

FromSource exhausts Typst's process-global file ids after 65,535 compilers #41

Description

@msallin

SystemWorld::new mints a file id per in-memory document (world.rs:108):

FileId::unique(RootedPath::new(
    VirtualRoot::Project,
    VirtualPath::new("<main>").expect("`<main>` is a valid virtual path"),
))

FileId::unique is a process-global NonZeroU16 interner (typst-syntax 0.15.1, path.rs:143):

let num = u16::try_from(interner.from_id.len() + 1)
    .and_then(NonZeroU16::try_from)
    .expect("out of file ids");
let leaked = Box::leak(Box::new(path));

Each compiler therefore leaks its RootedPath, and after 65,535 the expect panics inside create_compiler, which has no catch_unwind and so aborts the process. Upstream states the budget in a comment: "We can't leak more than 2^16 pairs".

FromSource and the static one-liners each create one, and those are the shapes the README leads with. A service rendering one document per request aborts after roughly 65k documents, and again after every restart.

FileId::unique is documented for ids created once per process, such as content read from stdin; typst-cli uses STDIN_ID: LazyLock<FileId>.

What I would like: allocate the <main> id once per process, or intern a fixed rooted path via RootedPath::new(...).intern(), which dedupes by path. Compilers sharing the id do not interfere, since slot maps are per world and comemo keys on the Source content hash. A regression test creating 70,000 compilers with system fonts disabled is cheap.

Happy to send a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions