[Mono.Android] Handle JNI failures during CoreCLR GC bridge processing

[simonrozsival]

Note

This pull request was created with the assistance of GitHub Copilot. The code and description were AI-generated and reviewed by the author.

Summary

Bridge processing in the CoreCLR/NativeAOT shared native host
(src/native/clr/host/bridge-processing.cc) called several JNI functions
without checking for failure. This could leave a Java exception pending —
making the next JNI call undefined behaviour (a CheckJNI abort on debug
builds) — or silently mis-handle an out-of-memory condition as a collected
object, tearing down a peer that is still alive. This follows the pattern
already used by GCBridge::trigger_java_gc and fails fast with a clear
diagnostic.

Changes

• add_reference / clear_references — check for a pending exception
  after the CallVoidMethod invocations of monodroidAddReference /
  monodroidClearReferences. The generated monodroidAddReference lazily
  allocates and appends to an ArrayList, so it can throw
  OutOfMemoryError under the same memory pressure that triggers a GC.
  Previously the exception was left pending (undefined behaviour on the next
  JNI call) and the keep-alive edge was silently dropped, risking premature
  collection of a still-referenced peer.

• take_weak_global_ref — NewWeakGlobalRef of a valid strong global
  reference only returns null when the VM is out of memory. The old code
  stored the null and then deleted the strong reference, losing the object
  (a subsequent NewGlobalRef of a null weak reference looks like a
  collected peer). A null result now fails fast.

• take_global_ref — a null NewGlobalRef result normally means the
  weak reference's target was collected, but it can also indicate a resource
  failure (null with a pending exception). Treating the latter as "collected"
  would tear down a live peer, so the two cases are now distinguished.

These are fault paths (out-of-memory, or a peer callback throwing); normal
bridge processing is unaffected.

Testing

Built the CoreCLR android-arm64 runtime with these changes and ran the full
Mono.Android.NET-Tests suite on an API 36 arm64-v8a emulator with
-p:UseMonoRuntime=false (CoreCLR):

• 915 passed, 0 failed, 55 skipped.
• With debug.mono.log=gc, the GC bridge ran 12 times during the suite,
  exercising the hardened add_reference / clear_references /
  take_global_ref / take_weak_global_ref paths, with no CheckJNI / JNI
  errors and none of the new fail-fast paths triggered (CheckJNI is active
  for the debuggable test app).
• One unrelated timing-sensitive test (JavaObjectTest.UnregisterFromRuntime,
  a registered-peer count assertion) flaked once under heavy GC-spew logging
  and passed on a clean run; it is not affected by these changes.

The new abort paths are only reachable under out-of-memory / a throwing peer
callback and were not exercised by fault injection.

Note

This touches the same file as #12040 (GC bridge JNI-overhead cleanup); the two
changes are independent but edit adjacent code, so a trivial merge conflict is
expected depending on merge order.

[Copilot]

Pull request overview

Hardens CoreCLR/NativeAOT GC bridge processing against JNI failure modes by failing fast (with diagnostics) when peer callbacks or reference promotions leave a pending Java exception or indicate an OOM/resource failure. This prevents undefined JNI behavior and avoids incorrectly treating resource failures as “object collected,” which could tear down still-live peers.

Changes:

• Add abort_on_pending_java_exception() helper and use it after monodroidAddReference / monodroidClearReferences JNI calls.
• Distinguish “weak target collected” vs “JNI failure w/ pending exception” when promoting weak refs via NewGlobalRef.
• Fail fast when NewWeakGlobalRef returns null (OOM/resource failure) to avoid losing the object.

Show a summary per file

File	Description
src/native/clr/include/host/bridge-processing-shared.hh	Adds a shared helper declaration for detecting/logging pending Java exceptions during bridge processing.
src/native/clr/host/bridge-processing.cc	Implements fail-fast exception handling and tightens null-ref handling for global/weak-global ref transitions.

Copilot's findings

• Files reviewed: 2/2 changed files
• Comments generated: 1