Skip to content

fix: stabilize Hermes external memory tests - #1529

Open
mrousavy wants to merge 1 commit into
mainfrom
fix/hermes-external-memory-test-flake
Open

fix: stabilize Hermes external memory tests#1529
mrousavy wants to merge 1 commit into
mainfrom
fix/hermes-external-memory-test-flake

Conversation

@mrousavy

@mrousavy mrousavy commented Aug 25, 2026

Copy link
Copy Markdown
Member

Summary

  • resample Hermes external-memory measurements when process-wide GC activity contaminates a snapshot
  • retain native ArrayBuffers across measurement attempts so Nitro allocations cannot be collected
  • cover both HybridObject and ArrayBuffer accounting with the same strict helper
  • throw with external-byte and GC-count diagnostics when Hermes instrumentation is unavailable or no exact sample is observed

Root cause

The failed iOS ASan job expected a 1 MiB increase but observed 1 MiB minus 1 KiB. The immediately preceding test creates a 1 KiB native ArrayBuffer and releases it. ASan changed timing enough for Hermes to collect that unrelated buffer between the global js_externalBytes snapshots. This was a test measurement race, not an AddressSanitizer violation or missing Nitro accounting.

Validation

  • bun typecheck
  • bun --cwd example lint-ci
  • local iOS ASan build
  • local full iOS ASan harness: 544/544
  • local focused external-memory iOS ASan harness: 4/4, repeated 6 times
  • hosted iOS ASan harness: 544/544, passed 3 consecutive executions
  • hosted iOS TSan and Default harnesses: 544/544
  • complete PR matrix green, including Android ASan and Default
  • original failing ASan job passed when rerun unchanged, confirming the timing-dependent flake

@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
nitro-docs Skipped Skipped Aug 25, 2026 9:35pm

Request Review

@mrousavy
mrousavy marked this pull request as ready for review August 25, 2026 22:24

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 68fbf4699a

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread example/src/getTests.ts
Comment on lines +114 to +115
if (externalMemoryDelta === expectedSize) {
return externalMemoryDelta

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject samples that span garbage collection

When Hermes runs a GC between these snapshots, an exact net delta is still contaminated: the immediately preceding ArrayBuffer test leaves an eligible 1 KiB native buffer, so a new allocation over-reported by 1 KiB could coincide with that buffer's collection and produce the expected delta. Because garbageCollections is only logged, this condition would accept the sample and mask the accounting regression; require both the exact delta and zero intervening collections, otherwise retry.

Useful? React with 馃憤聽/ 馃憥.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant