Document and implement the CIEL release flow in CIEL Lab (Release > Deploy → QA Bulk Validation → Export version)

[filiperochalopes]

User story

As a release maintainer, I want a documented and guided release flow in CIEL Lab so I can run QA, deploy a version, and generate exports with clear gating and outcomes.

Use case

Start a release from Release > Deploy, run QA gating, create a version when allowed, and generate exports via Release > Export version with background processing and clear status.

Requirements

• Provide a release flow with phases:
  • Configuration
  • QA
  • Deploy
  • Export
• Configuration phase:
  • Input for version name
  • Default version name format: v[yyyy-mm-dd]
• QA phase:
  • Run QA checks against CIEL/HEAD
  • If any QA error exists, stop the flow and do not proceed
  • Keep the QA report available at /qa/bulk-validation (HEAD source)
• Deploy phase:
  • If there are only warnings (or lower severity), allow proceeding to create the version via OCL API
• Export phase:
  • After creating the version, poll export availability every 10 minutes
  • Abort if export is not available after 72 attempts, and mark the release as incomplete
  • Once export is available:
    • Run QA bulk-validation for the export
    • If no errors, proceed with export scripts for:
      • SQL - CIEL Admin (OpenMRS v1.11)
      • mysqldump from CIEL Admin v2+?
      • Test: run Liquibase scripts from the new mysqldump (optional test step)
• Storage and lifecycle:
  • Store exported files temporarily in local RustFS (S3-like bucket)
  • Make downloads available for 21 days
  • Automatically remove exports after 21 days to avoid consuming local storage
• UI flow and orchestration:
  • The process starts in Release > Deploy
  • It runs in background, triggering steps in order across:
    • Release > QA Bulk Validation
    • Release > Export version
  • Export must be accessible from a dedicated page: Release > Export version

Acceptance criteria

• A release can be started from Release > Deploy with a version name (default v[yyyy-mm-dd]).
• If HEAD QA ends with any error, the release stops and the report remains accessible at /qa/bulk-validation (HEAD).
• If HEAD QA has only warnings (or lower), the system can create a version via OCL API.
• After version creation, the system polls export readiness every 10 minutes and aborts after 72 attempts if not available.
• When export becomes available, QA bulk-validation is run on the export and blocks export scripts if errors exist.
• Export artifacts are stored in RustFS, downloadable for 21 days, and automatically removed after that period.
• The user can access export results via Release > Export version.

[askanter]

@burke for next release this month do you think we could run this in parallel? One version based on a staging build and the standard production version? We would then compare sand maybe in May switch over to OCL only?

[filiperochalopes]

@bmamlin not @burke

[filiperochalopes]

@askanter We still need to assess whether all the required documentation, development, and testing will be ready by the April release date. I do not think so. A more realistic expectation may be a parallel release in May, followed by an OCL-only release in June.

That said, I am still not clear on how we would handle a parallel deployment if data is not actually being edited in both places. Right now, it seems edits are happening only in CIEL Admin, not in both OCL and CIEL Admin.

One possible approach, though, would be this: continue editing in CIEL Admin, then send everything done in May to OCL HEAD using @bmamlin script (but not create a version, just stage the data). That way, we would have data in both places and could run a parallel release test.

[askanter]

I was thinking of the latter. We wouldn't be testing the content editing on OCL, just that Burke would deploy the same content on staging and production, and that we'd run your release scripts from staging and compare against what we get from production via Burke's complete method. The question is whether getting the data into staging HEAD is worth it.