Detection

[NetZissou]

• Base YOLO Detector
• Camera-trap animal detection (closes Animal Detection #7 )
• Face detection (closes Face Detection #6 )

[egrace479]

A few notes and questions

[Copilot]

🟡 Changes recommended

Detector thresholds, class labeling, visualization, HDF5 discovery, and default job templates contain functional issues.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds YOLO-based face and animal detection pipelines for distributed image, Parquet, and HDF5 processing.

Changes:

• Adds detector classes, preprocessing, visualization, and result export.
• Adds SLURM/configuration templates and animal-detection documentation.
• Adds detection dependencies and MkDocs enhancements.

File summaries

File	Description
src/hpc_inference/utils/visualization.py	Adds detection visualization utilities.
src/hpc_inference/utils/transforms.py	Adds MegaDetector preprocessing.
src/hpc_inference/utils/__init__.py	Exports new utilities.
src/hpc_inference/inference/detection/face_detector.py	Implements face scoring.
src/hpc_inference/inference/detection/face_detect.py	Adds face-detection pipeline and CLI.
src/hpc_inference/inference/detection/base_detector.py	Defines shared YOLO detector behavior.
src/hpc_inference/inference/detection/animal_detector.py	Implements detailed animal detection.
src/hpc_inference/inference/detection/animal_detect.py	Adds animal-detection pipeline and CLI.
src/hpc_inference/inference/detection/__init__.py	Exports detection APIs.
scripts/face_detection/face_detect_parquet_template.slurm	Adds face/Parquet job template.
scripts/face_detection/face_detect_image_folder_template.slurm	Adds face/image job template.
scripts/face_detection/face_detect_hdf5_template.slurm	Adds face/HDF5 job template.
scripts/animal_detection/animal_detect_parquet_template.slurm	Adds animal/Parquet job template.
scripts/animal_detection/animal_detect_image_folder_template.slurm	Adds animal/image job template.
scripts/animal_detection/animal_detect_hdf5_template.slurm	Adds animal/HDF5 job template.
pyproject.toml	Adds detection extra and updates metadata.
mkdocs.yml	Registers detection guide and extensions.
main.py	Adds MkDocs file-inclusion macro.
docs/animal-detection-guide.md	Documents animal-detection workflows.
docs-requirements.txt	Adds MkDocs macros dependency.
configs/face_detection/config_face_detect_parquet_template.yaml	Adds face/Parquet configuration.
configs/face_detection/config_face_detect_image_folder_template.yaml	Adds face/image configuration.
configs/face_detection/config_face_detect_hdf5_template.yaml	Adds face/HDF5 configuration.
configs/embed/config_embed_parquet_template.yaml	Documents Parquet embedding configuration.
configs/embed/config_embed_image_folder_template.yaml	Documents image embedding configuration.
configs/animal_detection/config_animal_detect_parquet_template.yaml	Adds animal/Parquet configuration.
configs/animal_detection/config_animal_detect_image_folder_template.yaml	Adds animal/image configuration.
configs/animal_detection/config_animal_detect_hdf5_template.yaml	Adds animal/HDF5 configuration.

Review details

Suppressed comments (2)

src/hpc_inference/inference/detection/face_detect.py:317

• store_true combined with default=True makes this CLI flag a no-op and gives command-line users no way to disable even distribution. Expose the inverse option as well.

    parser.add_argument("--evenly_distribute", action="store_true", default=True,
                        help="Distribute files evenly based on size (recommended for better load balancing)")

docs/animal-detection-guide.md:413

• This path misspells the working directory created earlier, so the copy-and-paste command fails unless an unrelated directory exists.

cd ~/animal_detetction

• Files reviewed: 26/29 changed files
• Comments generated: 23
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Profiling threads can prevent failed jobs from exiting, and several documented input/output contracts remain inaccurate or incomplete.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (13)

Previously missed (11) — in code that hasn't changed since the last review.

src/hpc_inference/inference/detection/animal_detect.py:220

• This starts a non-daemon thread whose loop only exits when usage_stop is set, but that happens only after the entire inference/save loop succeeds. Any DataLoader, CUDA, model, or Parquet exception leaves the thread alive and can keep the failed SLURM task running until its wall-time limit. Wrap the processing section in try/finally and always set the event and join the thread in finally.
  src/hpc_inference/inference/detection/face_detect.py:204
• This starts a non-daemon thread whose loop only exits when usage_stop is set, but that happens only after the entire inference/save loop succeeds. Any DataLoader, CUDA, model, or Parquet exception leaves the thread alive and can keep the failed SLURM task running until its wall-time limit. Wrap the processing section in try/finally and always set the event and join the thread in finally.
  src/hpc_inference/utils/transforms.py:136
• The public color argument is an RGB tuple, but only color[0] is passed to F.pad, so non-gray padding such as (255, 0, 0) becomes white in every channel. Either apply the corresponding value per channel or change the API to accept only a scalar; the current implementation does not honor its documented input.
  configs/animal_detection/config_animal_detect_hdf5_template.yaml:126
• This field is computed across animal, person, and vehicle detections, so it can be nonzero when no animal was detected. Change the parenthetical to “0.0 if no detections” to match the emitted data.
  configs/animal_detection/config_animal_detect_image_folder_template.yaml:104
• This field is computed across animal, person, and vehicle detections, so it can be nonzero when no animal was detected. Change the parenthetical to “0.0 if no detections” to match the emitted data.
  configs/animal_detection/config_animal_detect_parquet_template.yaml:119
• This field is computed across animal, person, and vehicle detections, so it can be nonzero when no animal was detected. Change the parenthetical to “0.0 if no detections” to match the emitted data.
  docs/animal-detection-guide.md:461
• This field is computed across animal, person, and vehicle detections, so it can be nonzero when no animal was detected. Change the parenthetical to “0.0 if no detections” to match the emitted data.

This issue also appears in the following locations of the same file:

• line 485
• line 520
  mkdocs.yml:13

• The PR claims to close issue #6, whose remaining acceptance item is documentation for using the face-detection job script, but this navigation update adds only an animal-detection guide and the PR contains no corresponding face guide. Add and link face-detection usage documentation before closing that issue.
  scripts/animal_detection/animal_detect_hdf5_template.slurm:91
• This field is computed across animal, person, and vehicle detections, so it can be nonzero when no animal was detected. Change the parenthetical to “0.0 if no detections” to match the emitted data.
  scripts/animal_detection/animal_detect_image_folder_template.slurm:96
• This field is computed across animal, person, and vehicle detections, so it can be nonzero when no animal was detected. Change the parenthetical to “0.0 if no detections” to match the emitted data.
  scripts/face_detection/face_detect_parquet_template.slurm:86
• This schema says the image column contains encoded JPEG/PNG bytes, but the pipeline passes rows to decode_image, which only accepts a raw height*width*3 uint8 BGR buffer plus a matching size column. Following this template produces rows that are rejected and silently skipped. Document the raw-buffer schema used by the new config template.

docs/animal-detection-guide.md:525

• This runnable example uses Image.open without importing Image in this or any preceding Python snippet, so copying the documented workflow raises NameError. Add the PIL import alongside the visualization import.

from hpc_inference.utils.visualization import plot_detections_matplotlib

image_uuid = first_detection['uuid']
image_path = f"camera_trap_data/data/test/ENA24/{image_uuid}"

image = Image.open(image_path)

docs/animal-detection-guide.md:488

• num_detections includes every MegaDetector class, including people and vehicles, so this reports a person-only image as one where an animal was found. Parse the detection JSON and test for the animal class, or relabel the metric as images with any detection.

# View summary
print(f"Processed {len(df)} images")
print(f"Found animals in {sum(df['num_detections'] > 0)} images")
print(f"Average detections per image: {df['num_detections'].mean():.2f}")

• Files reviewed: 27/29 changed files
• Comments generated: 2
• Review effort level: Balanced

[NetZissou]

@thompsonmj could you review this PR?

---

Recap

This PR adds batch animal detection (MegaDetector) and face detection (YOLO-face) as a new hpc_inference.inference.detection module, built on the same distributed pattern as the embedding pipeline: one SLURM task per GPU, files split across ranks, results written as Parquet per rank.

• Three input types for both detectors: image folders, Parquet, and HDF5 (the latter via the July HDF5 integration).
• Detector classes (BaseDetector -> AnimalDetector, FaceDetector) wrapping Ultralytics; animal detection outputs per-image bounding boxes, class, and confidence (JSON column), face detection outputs a max face-confidence score per image.
• Preprocessing via a letterbox transform adapted from Microsoft's Pytorch-Wildlife (MIT), plus a visualization utility to draw boxes back on the original image.
• Templates and docs: six SLURM + six config templates (animal/face × images/parquet/hdf5), a new [detection] extra in pyproject.toml (Python >= 3.10), and an end-to-end Animal Detection Guide using the ENA24 camera-trap subset.

---

Already covered

We ran a comprehensive Copilot review last week and addressed all suggestions in latest commits. After that, tested the workflow end to end on Cardinal, all passing:

the guide's single- and multi-process scripts on an image folder, plus the shipped hdf5/parquet templates for both animal and face detection, using the ENA24 camera-trap subset (1,120 images) converted into all three input formats (parquet, image folder, h5). Results are consistent across input paths. If you'd like to try it yourself, everything is staged on scratch: xxx/TreeOfLife/analytics/detection_dryrun/

---

Review scope

What I'd like your review to cover:

1. The guide (docs/animal-detection-guide.md), read as a first-time user: is the flow clear and complete, and is the MegaDetector / Pytorch-Wildlife attribution and citation wording right?

2. The core design, not line by line: the detector API and output schema in base_detector.py / animal_detector.py / face_detector.py,and the pyproject.toml changes ([detection] extra, Python >= 3.10).