feat: implement dispose() for deterministic native resource release - #198
Open
jslok wants to merge 2 commits into
Open
feat: implement dispose() for deterministic native resource release#198jslok wants to merge 2 commits into
jslok wants to merge 2 commits into
Conversation
Two related delegate-lifecycle fixes in createModel / HybridTfliteModel: 1. Skip null delegates: delegate factories can legitimately return nullptr (e.g. TfLiteCoreMlDelegateCreate on devices without a Neural Engine when enabled_devices is ANE-only). Registering that nullptr with TfLiteInterpreterOptionsAddDelegate crashes/corrupts the interpreter. Now a null delegate is skipped so the model falls back to CPU, and getDelegates() reports only the delegates that were actually registered. 2. Free delegates: TFLite's C API does not transfer delegate ownership to the interpreter - the caller must delete delegates itself after the interpreter is destroyed. They were never freed, so every model destruction leaked the delegate's compiled kernels / driver contexts (GPU: TfLiteGpuDelegateV2Delete, NNAPI: TfLiteNnapiDelegateDelete, CoreML: TfLiteCoreMlDelegateDelete). Deleters run after TfLiteInterpreterDelete (delegates must outlive the interpreter), and also on the interpreter-creation failure path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HybridTfliteModel inherits Nitro's default no-op dispose(), so JS calling
model.dispose() frees nothing - the interpreter, delegates and model buffer
only go away when GC drops the last reference. Worklet runtimes (frame
processors) may not GC for a long time, especially while the app is
backgrounded, so multi-hundred-MB models and their GPU contexts stay
resident with no way to release them deterministically.
This implements a real dispose():
- Frees interpreter -> delegates -> model buffer immediately (same order the
destructor uses; the shared releaseNativeResources() nulls pointers so
destructor-after-dispose is a no-op).
- Thread-safe via a lifecycle mutex: dispose() blocks until an in-flight
inference on another thread completes - freeing the interpreter under a
running TfLiteInterpreterInvoke would be a native crash. The mutex is
uncontended in normal operation (~ns per lock vs ~ms per inference).
- run() re-checks disposal on the async thread, since dispose() may land
between the caller-thread input copy and the async invoke.
- All post-dispose calls (runSync/run/getInputs/getOutputs) throw a
catchable JS error ('TFLite: Model was disposed!') instead of crashing.
- Idempotent; also releases resources on the AllocateTensors failure path
in the constructor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
HybridTfliteModelinherits Nitro's default no-opdispose(), somodel.dispose()from JS frees nothing — the interpreter, delegates, and model buffer only go away when GC drops the last reference.That's a real problem for frame-processor use: worklet runtimes may not GC for a long time (especially while the app is backgrounded), so large models + their GPU delegate contexts stay resident with no way to release them. In our app, releasing scanner models on background via a working
dispose()frees ~86 MB of native/GPU memory that otherwise just sits there.Implementation
dispose()frees interpreter → delegates → model buffer immediately, via a sharedreleaseNativeResources()that nulls pointers so destructor-after-dispose is a no-op. Idempotent.dispose()against inference —dispose()blocks until an in-flightTfLiteInterpreterInvokeon another thread (frame processor) completes, since freeing the interpreter mid-invoke is a native crash. The mutex is uncontended in normal operation (one ~ns lock per ~ms inference).run()re-checks disposal on the async thread —dispose()can land between the caller-thread input copy and the async invoke.runSync/run/getInputs/getOutputs) throwTFLite: Model was disposed!— a catchable JS error on any runtime, not a crash.AllocateTensorsfailure path in the constructor (previously leaked the interpreter+delegates when allocation failed).Testing
Running in production as a patch-package fix: models disposed on app background (with an active frame processor racing it), reloaded on foreground. No crashes across a soak test; post-dispose
runSynccalls surface as catchable JS errors.🤖 Generated with Claude Code