recipe(google/owlvit-base-patch32): add zero-shot-object-detection recipes

[codykk]

Recipe-only contribution (Effort L0). Adds fp32 and fp16 recipe configs for google/owlvit-base-patch32 (OwlViTForObjectDetection, task zero-shot-object-detection) on CPU. Goal L1 (perf) PASS on both precisions.

---

Model metadata

• What the model does: OWL-ViT (Vision Transformer for Open-World Localization) performs open-vocabulary zero-shot object detection — given an image and a set of text queries, it localizes and scores bounding boxes for arbitrary objects described by the text.
• Primary user stories: User supplies an image + text queries to obtain bounding boxes + confidence scores for described objects.
• Supported tasks: zero-shot-object-detection (winml-registered via ensure_hf_models_registered)
• Architecture:

OwlViTForObjectDetection
├── owlvit (OwlViTModel)
│   ├── text_model (OwlViTTextTransformer) — CLIP text encoder, 6 layers
│   └── vision_model (OwlViTVisionTransformer) — ViT-B/32, 12 layers, patch_size=32, image_size=768
├── class_head (OwlViTClassPredictionHead) — per-box class logits
└── box_head (OwlViTBoxPredictionHead) — per-box coordinates [cx, cy, w, h]

• Source/confidence: OwlViTForObjectDetection source from transformers + pinned checkpoint google/owlvit-base-patch32 config.json (verified).

---

Validation and support evidence

1. Baseline

• origin/main commit: 6ea1b070
• WinML version: 0.2.0
• Optimum probe: vendor: [], after_winml: ['feature-extraction', 'zero-shot-object-detection'], verdict: WINML-ONLY
• Build: PASS (33.3s, auto-config, no recipe)
• Perf: PASS (P50: 139.95ms, 6.90 sps, RAM +610.6MB)
• Eval: CLI-BLOCKED (winml eval does not support zero-shot-object-detection)
• Goal floor: L1 (baseline builds and perfs on CPU out-of-the-box)

2. Goal

• Effort: L0 (recipe-only, winml already registers owlvit for zero-shot-object-detection)
• Goal ceiling: L1 (perf)
  • L2: CLI-BLOCKED (no built-in numeric-parity command)
  • L3: CLI-BLOCKED (winml eval does not support zero-shot-object-detection task)
• Outcome: L0 (recipe + report)
• Goal success definition: winml build succeeds and winml perf runs without crash on all required (ep, device, precision) tuples.

3. Outcome

• Highest Goal verdict: L1 PASS (both tuples)
• Coverage: full (all cpu/cpu tuples PASS)
• Deferred tuples: none
• Shipped recipe paths:
  • examples/recipes/google_owlvit-base-patch32/cpu/cpu/zero-shot-object-detection_fp32_config.json
  • examples/recipes/google_owlvit-base-patch32/cpu/cpu/zero-shot-object-detection_fp16_config.json

4. Per-EP/device/precision results

Tier	EP / Device	Precision	Verdict	Mean	P50	Throughput	RAM Δ
L0	CPUExecutionProvider / cpu	fp32	PASS	—	—	—	—
L1	CPUExecutionProvider / cpu	fp32	PASS	145.74 ms	142.69 ms	6.86 sps	+610.5 MB
L0	CPUExecutionProvider / cpu	fp16	PASS	—	—	—	—
L1	CPUExecutionProvider / cpu	fp16	PASS	185.67 ms	187.76 ms	5.39 sps	+770.0 MB
L3	all	all	CLI-BLOCKED	—	—	—	—

L3 blocker: winml eval does not support task zero-shot-object-detection.

5. Delta

Recipe-vs-winml config diff:

Field (JSON pointer)	Auto-config value	Shipped recipe value	Reason
export.input_tensors order	[pixel_values, input_ids, attention_mask]	[input_ids, pixel_values, attention_mask]	Match forward() signature order per owlv2-003 finding

All other fields identical to auto-config output. Production recipe README (examples/recipes/README.md) remains untouched.

Reducibility: The input-order fix is a recipe-authoring rule (per-model declaration), not a class-of-models code gap — winml config generates the inputs in an arbitrary order; pinning to forward() order is a correctness choice specific to each checkpoint's recipe. Resolution: recipe (L0).

6. Analyze summary — component level and op level

winml analyze CLI-BLOCKED: runtime rule parquet files not found in the local environment. This is a host environment gap (missing rule data files), not a model or recipe issue. Analyze is not a Goal-tier requirement and does not block L0/L1 verdicts.

7. Reproduce commands

# fp32 build + perf
winml build -c examples/recipes/google_owlvit-base-patch32/cpu/cpu/zero-shot-object-detection_fp32_config.json \
  -m google/owlvit-base-patch32 -o $OUT/owlvit_cpu_fp32 --ep cpu --device cpu --rebuild
winml perf -m $OUT/owlvit_cpu_fp32/model.onnx --ep cpu --device cpu

# fp16 build + perf
winml build -c examples/recipes/google_owlvit-base-patch32/cpu/cpu/zero-shot-object-detection_fp16_config.json \
  -m google/owlvit-base-patch32 -o $OUT/owlvit_cpu_fp16 --ep cpu --device cpu --precision fp16 --rebuild
winml perf -m $OUT/owlvit_cpu_fp16/model.onnx --ep cpu --device cpu