Add optional PDF and image attachments to extract.ai

[Copilot]

Linked issue

What changes

Allow extract.ai to extract directly from PDFs and images, optionally combined with text, without Markdown conversion. This slice supports explicit local files through OpenAI Responses.

• Inputs: PDF, PNG, JPEG, and WebP descriptors with stable source IDs. Python batches use one attachment list per record; YAML supports literal paths or per-row path columns. Ordinary input paths remain text.
• Execution and caching: Bounded, immutable file snapshots; cache identity includes bytes, source order, image detail, text, and request settings. Preserves scoped credentials, retries, filtered row association, and saved-model outputs.
• Diagnostics: Per-attempt usage, timing, status, model/request IDs, and cache correlation—including incomplete and failed attempts. Missing usage remains unknown; reasoning is not counted twice. Credentials and binary payloads are excluded.

wrangles.extract.ai(
    "Extract the voltage rating and supporting page number.",
    api_key=api_key,
    attachments=[{"path": "/data/specification.pdf", "id": "datasheet"}],
    model="gpt-5.4",
    output={"voltage": "Voltage rating", "page": {"type": "integer"}},
    timeout=180, threads=1, retries=0, max_output_tokens=16000,
    store=False,
)

How it was verified

• 1,764 credential-safe tests passed; 137 intentionally deselected.
• 297 focused tests passed, covering payload bytes, source/row association, cache invalidation, credential isolation, validation, and incomplete-attempt accounting.
• Normal recipe CLI verified actionable missing-file errors without exposing credentials.
• Live PDF/image/mixed trials and RSGroup SF_AMF60 remain outstanding: credentials and customer fixtures were unavailable.
• Final automated review/security validation hit its service time limit. CI requires approval.

Compatibility and risk

Text-only behavior, return shapes, schemas, and configuration precedence remain unchanged. Attachments require a vision-capable structured-output Responses model; URLs, raw bytes, provider file IDs, and Chat Completions are unsupported.

Files are sent inline—no separate upload lifecycle. Existing store: true defaults apply. Visual requests can require larger timeouts/output budgets and lower concurrency. Source references remain model-generated claims requiring validation.

Rollback: omit attachments to restore text-only calls, or revert the change.

Ready-for-review checklist

• One human delivery owner is assigned
• The linked issue and intended milestone are correct
• The branch is current with main and has no merge conflicts
• Focused tests pass
• New or changed behavior has direct test coverage
• Documentation/schema/configuration is updated where applicable
• The PR contains no unrelated changes
• The PR description reflects the branch's current scope and latest validation
• One primary reviewer is requested only when this PR is ready

See the pull request workflow.

• Fixes Add optional PDF and image inputs to extract.ai #1178

[ebhills]

@copilot - Can we wire in s3 (using our connector and/or boto) so that we can reference remote files?

[Copilot]

@copilot - Can we wire in s3 (using our connector and/or boto) so that we can reference remote files?

Implemented in f0ed9b8, with real-botocore regression tests in c8b0194.

Use attachments=[{"path": "s3://bucket/specification.pdf", "id": "datasheet"}]; YAML literal paths and per-row attachment columns accept the same URI. It uses existing boto3 with IAM/profile/environment credentials, bounded downloads, and explicit stream/client cleanup. Objects are re-read before cache lookup so changed content or revoked access cannot reuse stale results. For explicit credentials/custom endpoints, the existing s3.download_files connector remains available.

Checks: 318 focused tests, 1,896 local-suite tests, package builds, and secret scans passed; CodeQL found 0 alerts. Final targeted review found no significant issues; the automated review binary was unavailable. Live S3/OpenAI validation remains outstanding.

Recommended disposition: Comment only

Next steps

1. PR assignee: Approve CI, run the authorized S3 example in docs/extract_ai_configuration.md, and request review.
2. Reviewer: Verify S3 input, cache, and credential behavior; approve after required checks pass.