fix: skip unavailable delegates and free delegates on model destruction - #197
Open
jslok wants to merge 1 commit into
Open
fix: skip unavailable delegates and free delegates on model destruction#197jslok wants to merge 1 commit into
jslok wants to merge 1 commit into
Conversation
Two related delegate-lifecycle fixes in createModel / HybridTfliteModel: 1. Skip null delegates: delegate factories can legitimately return nullptr (e.g. TfLiteCoreMlDelegateCreate on devices without a Neural Engine when enabled_devices is ANE-only). Registering that nullptr with TfLiteInterpreterOptionsAddDelegate crashes/corrupts the interpreter. Now a null delegate is skipped so the model falls back to CPU, and getDelegates() reports only the delegates that were actually registered. 2. Free delegates: TFLite's C API does not transfer delegate ownership to the interpreter - the caller must delete delegates itself after the interpreter is destroyed. They were never freed, so every model destruction leaked the delegate's compiled kernels / driver contexts (GPU: TfLiteGpuDelegateV2Delete, NNAPI: TfLiteNnapiDelegateDelete, CoreML: TfLiteCoreMlDelegateDelete). Deleters run after TfLiteInterpreterDelete (delegates must outlive the interpreter), and also on the interpreter-creation failure path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two related delegate-lifecycle bugs in
createModel/HybridTfliteModel:1. Null delegates are registered with the interpreter
Delegate factories can legitimately return
nullptr— e.g.TfLiteCoreMlDelegateCreateon devices without a Neural Engine (the defaultenabled_devicesis ANE-only).createModelpassed thatnullptrstraight intoTfLiteInterpreterOptionsAddDelegate, corrupting/crashing interpreter creation.Fix: skip null delegates so the model falls back to CPU.
getDelegates()now also reports only the delegates that were actually registered, instead of claiming a delegate is active when its creation returned null.2. Delegates are never freed
TFLite's C API does not transfer delegate ownership to the interpreter — the caller must keep the delegate alive and delete it itself after the interpreter is destroyed (TFLite docs).
~HybridTfliteModeldeleted the interpreter but never the delegates, so every model destruction leaked the delegate's compiled kernels / driver contexts. For the GPU delegate that's a substantial native+driver allocation per model load — very visible in apps that load/unload models across their lifecycle.Fix: capture a per-delegate deleter (
TfLiteGpuDelegateV2Delete/TfLiteNnapiDelegateDelete/TfLiteCoreMlDelegateDelete) at creation time and run them in the destructor afterTfLiteInterpreterDelete(delegates must outlive the interpreter), plus on the interpreter-creation failure path.Testing
Running in production (React Native scanner app, Android GPU delegate + CoreML) as a patch-package fix — model load/release cycles no longer accumulate native memory, verified with heapprofd before/after.
🤖 Generated with Claude Code