Expose workflow metadata as a Pydantic model

[jeremyjordan]

Priority Level

Medium (Nice to have)

Is your feature request related to a problem? Please describe.

Workflow metadata is persisted as JSON but does not currently have a public, typed representation. This makes it harder for users and external tools to reliably inspect workflow status, stage configuration, outputs, and provenance.

Describe the solution you'd like

Add and publicly export a Pydantic WorkflowMetadata model, including an appropriate model for stage-level metadata.

The model should:

• Represent the existing workflow metadata format.
• Support metadata from running, failed, completed, completed-empty, and skipped stages.
• Provide consistent validation and JSON serialization.
• Preserve compatibility with existing workflow metadata files.
• Be importable from the public Data Designer interface.

This would establish a clear programmatic contract for workflow observability and integrations while keeping the workflow metadata format machine-readable and self-describing.

Describe alternatives you've considered

No response

Agent Investigation

No response

Additional context

No response

Checklist

• I've reviewed existing issues and the documentation
• This is a design proposal, not a "please build this" request

[andreatnvidia]

Implemented in #902. The PR adds public WorkflowMetadata and stage-level metadata models, validates metadata at read/write boundaries, and preserves compatibility with existing metadata files. I've also requested your review. Let us know if this is what you had in mind!