feat: wildlife re-ID pipeline for offline species identification#2
Open
JasonWildMe wants to merge 28 commits intomainfrom
Open
feat: wildlife re-ID pipeline for offline species identification#2JasonWildMe wants to merge 28 commits intomainfrom
JasonWildMe wants to merge 28 commits intomainfrom
Conversation
…nd design doc Three documents capturing the design for bringing Wildbook's MiewID algorithm to mobile for offline individual animal re-identification: - WILDLIFE_REID_FEASIBILITY.md: Platform evaluation and technical research - EMBEDDING_PACK_FORMAT.md: Detailed spec for the embedding pack zip format including Wildbook exporter implementation guide - plans/2026-02-25-wildlife-reid-design.md: Full approved design covering fork strategy, data model, ML pipeline, sync protocol, screens, and MVP scope Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Detailed bite-sized implementation plan covering 6 phases: - Phase 0: Project setup (ONNX Runtime, type definitions) - Phase 1: Core services (ONNX inference, embedding matching, pack manager) - Phase 2: State management (wildlife Zustand store) - Phase 3: Pipeline orchestration (detect → embed → match → save) - Phase 4: Navigation and screen scaffolding (8 screens) - Phase 5: Integration and end-to-end wiring - Phase 6: Testing, E2E flows, and cleanup Each task follows TDD with exact file paths, code, test commands, and commit steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ch, pack manager) - OnnxInferenceService: wraps onnxruntime-react-native for model loading, detection stubs, and embedding extraction stubs - EmbeddingMatchService: cosine similarity computation with top-N matching against multi-embedding database entries - PackManager: embedding pack lifecycle (init dirs, load manifest, parse binary embeddings, manage individuals, delete packs) - Add onnxruntime-react-native mock to jest.setup.ts - Add barrel exports in services/index.ts - Fix pre-existing test timeouts in ModelDownloadScreen and PassphraseSetupScreen (increase per-test timeout to 20s) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The JS/TS section was running `npm test` which chains jest, test:android, and test:ios. Native tests are already run by their own hook sections when Swift/Kotlin files are staged. Running them redundantly in the JS/TS section causes failures when native toolchains are unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Also fixes flaky test timeouts under WSL2: - Increase Jest testTimeout from 10s to 60s - Limit pre-commit test workers to 50% of CPUs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ndividuals Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ky failures - Add accuracy field to ProcessPhotoParams.gps for consistency with Observation type - Fix jest.setup.ts setTimeout(10000) overriding jest.config.js testTimeout(60000) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace chat/models navigation with wildlife-specific navigation: 4 tabs (Home, Packs, Observations, Sync) plus root-level stack screens for Capture, DetectionResults, MatchReview, ObservationDetail, PackDetails, and Settings. Legacy navigation types preserved for existing screens until Task 6.4 cleanup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the PacksScreen placeholder with a full implementation that lists downloaded embedding packs from the wildlife Zustand store. Each pack card shows species display name, individual count, formatted export date, and storage size. Tapping a pack navigates to PackDetails. Empty state shows a "No Packs Downloaded" message. Includes 17 RNTL tests covering empty state, card rendering, multiple packs, navigation, and testIDs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the CaptureScreen placeholder with a full directory-based screen that allows users to take or select a photo, runs the wildlife detection pipeline, saves the observation to the store, and navigates to the DetectionResults screen. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace placeholder ObservationsScreen with full implementation featuring filter bar (All, Pending, Reviewed, Synced), FlatList of observations with thumbnails, timestamps, detection counts, and review status. Adds 19 RNTL tests covering rendering, filtering, navigation, and empty state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… observations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded device info with Platform.OS/Version and add getDeviceLocation helper (returns null for now) so GPS flows through the pipeline and observation consistently. Tests verify both fields. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When approving a match to an existing local individual, call addEmbeddingToLocalIndividual to add the detection's embedding and cropped image URI to the individual's profile, building a richer representation over time. Pack individuals are unaffected. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ading Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hat) Remove all legacy Off Grid AI chat modules that are no longer used by the wildlife re-identification app. This includes: - Services: LLM, image generation, voice, whisper, tools, model manager, active model service, intent classifier, document/PDF extraction, background download, HuggingFace browser, CoreML browser - Screens: ChatScreen, HomeScreen, ModelsScreen, ModelSettingsScreen, DownloadManagerScreen, GalleryScreen, Projects screens, VoiceSettings, StorageSettings, DeviceInfo, ModelDownload - Components: ChatInput, ChatMessage, ModelCard, ModelSelectorModal, GenerationSettingsModal, MarkdownText, VoiceRecordButton, DebugSheet, ProjectSelectorSheet, ToolPickerSheet, ThinkingIndicator - Stores: chatStore, projectStore, whisperStore - Hooks: useVoiceRecording, useWhisperTranscription - Types: whisper.rn.d.ts, legacy model/chat/image types - Constants: models.ts, LLM-specific constants - Utils: coreMLModelUtils, messageContent - All associated tests (contracts, integration, unit, RNTL) Updates: - App.tsx simplified to only initialize pack manager and hardware - appStore trimmed to theme, onboarding, and device info - SettingsScreen shows only Security settings - Navigation types cleaned of legacy routes - OnboardingScreen navigates to Main after completion - hardware.ts stripped of model recommendation code - Barrel exports updated for all modules - Test helpers and factories stripped of legacy references - Coverage thresholds adjusted (70/60/70/70) for post-strip baseline Kept: wildlife pipeline, ONNX inference, embedding match, pack manager, auth, hardware service, all wildlife screens and tests. 29 test suites, 408 tests passing. TypeScript compiles cleanly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… cleanup) - Add statusSuccess/Warning/Error to theme palettes for dark mode support - Replace hardcoded hex colors in BoundingBoxOverlay, SyncScreen, CandidateCard - Fix critical DetectorConfig empty cast with loadDetectorConfig + safe defaults - Remove unused gps field from ProcessPhotoParams (kept in Observation) - Add TODO(P0) comments to ONNX inference stubs - Type CandidateCard styles prop properly - Update tests with required mocks for packManager and embeddingDatabaseBuilder Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Register SecuritySettings route in root navigator (P1: tapping Security in Settings previously dispatched unhandled nav action) - Add PackDetailScreen showing pack species, feature class, individual count - Point PackDetails route to PackDetailScreen instead of PacksScreen (P2: tapping a pack previously just pushed another list copy) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a complete offline wildlife re-identification pipeline to Off Grid Mobile, enabling field researchers to photograph animals, detect individuals via YOLO object detection, generate MiewID v4 embeddings, and match against pre-built species embedding packs — all without internet connectivity.
Key additions:
Type of Change
Screenshots / Screen Recordings
N/A — no device testing yet; screens are structurally complete but require real ONNX inference wiring before visual validation.
Checklist
General
Testing
npm test) — 408 tests, 29 suitesReact Native Specific
SPACING/TYPOGRAPHYconstants from the themeuseThemedStylespattern (not inline or staticStyleSheet.create)FlatList/FlashList(not.map()insideScrollView)Performance & Models
/vs\\)Security
Related Issues
This implements the wildlife re-ID feasibility plan documented in
docs/plans/2026-02-25-wildlife-reid-implementation.md.Additional Notes