Skip to content

Retain runner artifacts with scoped native object storage - #82

Draft
ancongui wants to merge 1 commit into
integration/crm-recovery-editorialfrom
feat/native-object-storage
Draft

ancongui wants to merge 1 commit into
integration/crm-recovery-editorialfrom
feat/native-object-storage

Conversation

@ancongui

Copy link
Copy Markdown
Contributor

Runner artifact bytes currently depend on retained local files and complete archive bundles. This adds a scoped object-storage publication path using native PyFly ECM providers, independent PostgreSQL chunk rows, constant-size manifest roots, keyset reads and explicit reconciliation after uncertain writes. Raw-file upload, lease-checked streaming read and private atomic restore are wired into the controller and artifact producer/consumer paths. Existing local bindings remain unchanged unless explicitly configured.

Validation: 60 host cases passed (one existing process-dependent legacy case excluded under the effects-denial guard); two dedicated PostgreSQL cases passed in 5.37s using a non-superuser application role, including native DI/ASGI upload, concurrent publication, paged restore and lost-reply readback. The owned fixture stopped and temporary credentials were removed. Full mypy checked 414 source files; Ruff/format passed; strict documentation, 10 docs units and 73 pages/10,359 local links passed. No models, Docker workloads, cloud services or Original data were used.

This is a draft integration slice. Live provider acceptance, released native GCS support, Kubernetes scheduling, a reference-native Docker carrier, setup UI, catalog-to-file content reads and production retention remain open. The legacy carrier still buffers archives and RunnerPoolWorker still depends on its admitted central workspace. No enterprise-scale acceptance or migration is claimed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant