Skip to content

Optimize JS compiler caching for standard builds#27026

Open
brendandahl wants to merge 1 commit into
emscripten-core:mainfrom
brendandahl:cache-fix
Open

Optimize JS compiler caching for standard builds#27026
brendandahl wants to merge 1 commit into
emscripten-core:mainfrom
brendandahl:cache-fix

Conversation

@brendandahl
Copy link
Copy Markdown
Collaborator

Linker system stubs, such as libpthread_stub.js, are dynamically appended to settings.JS_LIBRARIES for all standard C/C++ builds. Previously, the JS output cache check did not distinguish between these system stubs and user-defined JS libraries (passed via --js-library). As a result, standard builds always bypassed the compilation cache, triggering a Node.js compilation run on every invocation.

Restrict the cache bypass condition to check only for user libraries (libraries located outside the Emscripten source tree). Standard compilations now successfully hit the JS output cache, decreasing baseline compilation wall clock time of a hello world C program by 52% (from 472 ms down to 282 ms).

Add an integration test to validate JavaScript compilation caching (covering hits, misses, user library bypasses, and option-based cache entries).

@brendandahl brendandahl requested review from kripken and sbc100 May 28, 2026 22:11
Comment thread tools/emscripten.py
# Sadly we have to skip the caching whenever we have user JS libraries. This is because
# these libraries can import arbitrary other JS files (either vis node's `import` or via #include)
if DEBUG or settings.BOOTSTRAPPING_STRUCT_INFO or config.FROZEN_CACHE or settings.JS_LIBRARIES:
has_user_libs = any(not lib.startswith(utils.path_from_root('src/')) for lib in settings.JS_LIBRARIES)
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could stick system libs in a separate setting, but that seems like a bigger change.

Copy link
Copy Markdown
Collaborator

@sbc100 sbc100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Comment thread test/test_sanity.py Outdated
Comment thread test/test_sanity.py Outdated
Linker system stubs, such as libpthread_stub.js, are dynamically
appended to settings.JS_LIBRARIES for all standard C/C++ builds.
Previously, the JS output cache check did not distinguish between
these system stubs and user-defined JS libraries (passed via
--js-library). As a result, standard builds always bypassed the
compilation cache, triggering a Node.js compilation run
on every invocation.

Restrict the cache bypass condition to check only for user
libraries (libraries located outside the Emscripten source tree).
Standard compilations now successfully hit the JS output cache,
decreasing baseline compilation wall clock time of a hello world C
program by 52% (from 472 ms down to 282 ms).

Add an integration test to validate JavaScript
compilation caching (covering hits, misses, user library bypasses,
and option-based cache entries).
Copy link
Copy Markdown
Member

@kripken kripken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants