From 2678e847fb47cf8499e32dd8d622f85521b9be16 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 7 Jul 2026 07:55:43 +0000 Subject: [PATCH 1/2] =?UTF-8?q?feat(gitops-api):=20F7=20=E2=80=94=20pin=20?= =?UTF-8?q?higher-level=20KRM=20objects=20as=20first-class=20documents?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Flux HelmRelease, Argo CD Application, and KRO resources already mirror and edit through the kind-agnostic pipeline exactly like core resources (no per-kind allowlist; only a small deny list of noisy built-ins). F7 makes that guarantee load-bearing rather than assumed. No product code changes. - manifestedit corpus: helmrelease.yaml, argocd-application.yaml, kro-podinfoapp.yaml flow through the existing globbed round-trip (byte-identical) and convergence (perturb-then-settle) gates. - e2e (manager,f7-higher-level-krm): a Flux HelmRelease is mirrored to its canonical path and a live chart-version bump round-trips in place, preserving a hand-authored comment. Uses the Flux CRDs already in the base e2e cluster; spec.suspend keeps the helm-controller inert. - docs: f7 design doc + user guide installing-apps-as-krm.md (install an app = add a KRM document; the chart-inflation boundary stays refused); README ladder marks F7 implemented. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/README.md | 2 + docs/design/gitops-api/README.md | 6 +- .../f7-higher-level-krm-documents.md | 159 ++++++++++++++++ docs/installing-apps-as-krm.md | 126 +++++++++++++ .../testdata/corpus/argocd-application.yaml | 18 ++ .../testdata/corpus/helmrelease.yaml | 21 +++ .../testdata/corpus/kro-podinfoapp.yaml | 11 ++ test/e2e/helmrelease_mirror_edit_e2e_test.go | 176 ++++++++++++++++++ test/e2e/templates/manager/helmrelease.tmpl | 19 ++ .../manager/watchrule-helmrelease.tmpl | 11 ++ 10 files changed, 548 insertions(+), 1 deletion(-) create mode 100644 docs/design/gitops-api/f7-higher-level-krm-documents.md create mode 100644 docs/installing-apps-as-krm.md create mode 100644 internal/git/manifestedit/testdata/corpus/argocd-application.yaml create mode 100644 internal/git/manifestedit/testdata/corpus/helmrelease.yaml create mode 100644 internal/git/manifestedit/testdata/corpus/kro-podinfoapp.yaml create mode 100644 test/e2e/helmrelease_mirror_edit_e2e_test.go create mode 100644 test/e2e/templates/manager/helmrelease.tmpl create mode 100644 test/e2e/templates/manager/watchrule-helmrelease.tmpl diff --git a/docs/README.md b/docs/README.md index 4331f3d7..69654333 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,6 +11,8 @@ If you only want the supported product docs, start with the files below. - [`../README.md`](../README.md): product overview and end-to-end quick start - [`configuration.md`](configuration.md): core configuration objects and how they fit together +- [`installing-apps-as-krm.md`](installing-apps-as-krm.md): installing an app is adding a KRM + document — Flux `HelmRelease`, Argo CD `Application`, KRO, and core resources all mirror and edit alike - [`commit-signing.md`](commit-signing.md): how valid Git signatures map to platform verification - [`github-setup-guide.md`](github-setup-guide.md): GitHub repository and credential setup - [`sops-age-guide.md`](sops-age-guide.md): Secret encryption with SOPS + age diff --git a/docs/design/gitops-api/README.md b/docs/design/gitops-api/README.md index cccb5c1f..1c3684b0 100644 --- a/docs/design/gitops-api/README.md +++ b/docs/design/gitops-api/README.md @@ -124,7 +124,7 @@ base"), and the mirror-mode vs. intent-cluster topology — live in | # | Feature | Design doc | Status | |---|---------|-----------|--------| | F1 | Kustomize `images:` / `replicas:` edit-through — a live change produced by an override entry is written back to that entry, never through into the source manifest | [f1-images-replicas-edit-through.md](f1-images-replicas-edit-through.md) | implemented ([#198](https://github.com/ConfigButler/gitops-reverser/pull/198)) | -| F7 | Higher-level KRM objects as first-class documents — corpus + e2e pinning that Flux `HelmRelease`/`Kustomization`, Argo CD `Application`, KRO resources, and core resources mirror and edit like any KRM document (they should already; F7 *proves* it), plus "install an app = add KRM" user docs | — | not designed (expected small) — **launch-critical** | +| F7 | Higher-level KRM objects as first-class documents — corpus + e2e pinning that Flux `HelmRelease`/`Kustomization`, Argo CD `Application`, KRO resources, and core resources mirror and edit like any KRM document (they should already; F7 *proves* it), plus "install an app = add KRM" user docs | [f7-higher-level-krm-documents.md](f7-higher-level-krm-documents.md) | implemented (manifestedit corpus for HelmRelease/Application/KRO; HelmRelease mirror+edit e2e; [installing-apps-as-krm.md](../../installing-apps-as-krm.md) user docs) | | F2 | Render-root scoping — a GitTarget declares its render root (e.g. `overlays/env1`); base files reached through `../../base` become read-only context, dissolving overlay fan-out ambiguity. Launch scope: overlay `images:`/`replicas:` entries + overlay-local documents, shipped **with** the per-edit `FullyReflected` accounting | — | not designed — **launch-critical** | | F4 | New-file placement rules — sibling inference + `spec.placement` template so new resources land in the folder's convention, not the canonical REST path; includes creating the `resources:` entry when the target folder carries a kustomization | designed in [version2/gittarget-new-file-placement-rules.md](../manifest/version2/gittarget-new-file-placement-rules.md) | implemented (v1: declared policy + sibling inference steps 1/2/4 + kustomize `resources:` entry; step 3 and ordered-rule Option A deferred) | | F5 | Branch/session ergonomics — base-branch selection, opt-in remote branch cleanup, a GitTarget-level quiescence condition | — | not designed — **launch-critical** | @@ -143,6 +143,10 @@ base"), and the mirror-mode vs. intent-cluster topology — live in surface with terminal `Pushed=True` + SHA. - Refusals: unsupported kustomize features, duplicate identities, impure or foreign content — refuse-first, never mis-edit. +- Higher-level KRM documents (Flux `HelmRelease`, Argo CD `Application`, KRO + resources) mirror and edit exactly like core resources — the pipeline is + kind-agnostic, now pinned by F7's corpus + HelmRelease e2e + ([f7-higher-level-krm-documents.md](f7-higher-level-krm-documents.md)). ## Known boundary (what stays refused) diff --git a/docs/design/gitops-api/f7-higher-level-krm-documents.md b/docs/design/gitops-api/f7-higher-level-krm-documents.md new file mode 100644 index 00000000..962362ae --- /dev/null +++ b/docs/design/gitops-api/f7-higher-level-krm-documents.md @@ -0,0 +1,159 @@ +# F7: Higher-level KRM objects as first-class documents + +> Status: implemented — corpus + e2e pins land with this doc; **no product code +> changed** (the guarantee already held; F7 makes it load-bearing) +> Captured: 2026-07-07 +> Related: +> [README.md](README.md), +> [kustomize-support-boundary-and-product-model.md](kustomize-support-boundary-and-product-model.md), +> [f1-images-replicas-edit-through.md](f1-images-replicas-edit-through.md), +> [../../future/idea-application-editing.md](../../future/idea-application-editing.md), +> [../../architecture.md](../../architecture.md) + +## Premise: the pipeline is already kind-agnostic + +The launch pitch is "install an app = add a KRM document, roll out a version = +edit a field" and it must hold for the objects platform teams actually deploy: +Flux `HelmRelease`, Flux `Kustomization`, Argo CD `Application`, KRO resources, +and plain core objects. That already works, because **nothing in the write path +carries a per-kind allowlist**: + +- **Watch selection** is by `(group, version, resource)` resolved against the + *served* API surface — any served GVR (core, CRD, or aggregated) is watchable. + A `WatchRule`/`ClusterWatchRule` names plural resources and optional + `apiGroups`; an omitted group is resolved from discovery + ([../../../api/v1alpha3/watchrule_types.go](../../../api/v1alpha3/watchrule_types.go), + `ResourceRule` — "resolves the resource name across all served API groups"). + There is no set of "supported kinds." +- **The in-document editor** (`internal/git/manifestedit`) edits generic + `yaml.Node` trees keyed only by object identity + (apiVersion/kind/namespace/name). It has no notion of any specific kind + ([../../../internal/git/manifestedit/DECISION.md](../../../internal/git/manifestedit/DECISION.md)). +- **Placement** is GVR-driven: the cold-start default is + `{namespace}/{group}/{resource}/{name}.yaml`, and sibling inference / + `spec.placement` are equally generic + ([../../architecture.md](../../architecture.md), *What It Writes to Git*). + +The one kind-keyed *filter* in the pipeline is a small **exclusion** list of +noisy built-ins (`pods`, `events`, `endpoints`, `endpointslices`, `leases`, +`controllerrevisions`, `flowschemas`, `jobs`, …) — a deny list, never an allow +list ([../../../internal/watch/resource_policy.go](../../../internal/watch/resource_policy.go)). +`HelmRelease`, `Application`, and KRO kinds are not on it, so they are +default-allowed. The remaining kind-specific branches are **write-safety** and +**override** rules, and higher-level KRM falls through all of them: + +- Sensitive types (core `Secret` plus configured types) are encrypted before + they touch the worktree. A `HelmRelease`/`Application`/KRO instance is not + sensitive by default, so it writes plaintext like any document. +- The `/scale` subresource route and the kustomize `images:`/`replicas:` + override route ([F1](f1-images-replicas-edit-through.md)) apply to specific + shapes. A control-plane CR hits neither and takes the ordinary in-place patch + path. +- `status` is stripped from every mirrored object and status-only churn is + deduped, so a controller reconciling the CR does not produce spurious commits + (`internal/sanitize/marshal.go`, `internal/watch/target_watch.go`). + +This kind-agnosticism is **already e2e-proven for a generic CRD**: the +icecream-order lifecycle spec installs an arbitrary CRD and pins +create→file / update→file / delete→file-removed with `status` never committed +([../../../test/e2e/crd_lifecycle_e2e_test.go](../../../test/e2e/crd_lifecycle_e2e_test.go)). +What F7 adds is the same proof for a **named, real higher-level control-plane +type** (Flux `HelmRelease`) plus the un-installed kinds at the unit level. + +So a Flux `HelmRelease`, a Flux `Kustomization`, an Argo CD `Application`, a KRO +`ResourceGraphDefinition` or its generated instance, and a plain `Deployment` +all mirror and edit **identically** — they are KRM documents, and editing KRM +documents is the product surface (README, *"Higher-level products come into +scope as KRM, not as special cases"*). + +## Why prove something that already works + +F7 is not a code feature; it is the **proof and the promise**. It is +launch-critical for two reasons: + +1. **Regression fence.** "Kind-agnostic" is an *emergent* property of several + independent components (watch resolution, the node editor, placement, + sensitivity classification, override routing). A future special-case — a new + override heuristic, a placement tweak, a sensitivity default — could quietly + break a control-plane CR with no test noticing. A corpus + e2e make the + guarantee load-bearing so a regression announces itself. +2. **Marketing the boundary correctly.** The support contract is "we edit KRM + documents; we do not inflate Helm charts." Users conflate "Flux + `HelmRelease`" with "Helm." The user docs draw the line: the `HelmRelease` + *document* is first-class; `helmCharts:` *inflation* stays refused (README, + *The governing rule*; kustomize-support-boundary, *"No Helm"*). + +## What F7 pins + +### Coverage split + +| Kind | live e2e (mirror + edit) | unit corpus (round-trip + convergence) | +|---|---|---| +| Flux `HelmRelease` (`helm.toolkit.fluxcd.io/v2`) | ✅ CRD in base e2e cluster | ✅ | +| Argo CD `Application` (`argoproj.io/v1alpha1`) | — (CRD not in cluster) | ✅ | +| KRO instance, e.g. `PodInfoApp` (`kro.run` group) | — (KRO is demo-only) | ✅ | +| core (`ConfigMap`/`Deployment`) | already pinned (inplace-edit, f4-placement, scale) | already pinned | + +Flux CRDs (`helmreleases.helm.toolkit.fluxcd.io`, +`kustomizations.kustomize.toolkit.fluxcd.io`) are established during standard +e2e setup ([../../../test/e2e/Taskfile.yml](../../../test/e2e/Taskfile.yml), +`_flux-installed`), so the live pin uses a `HelmRelease`. The Argo and KRO +controllers are not in the base cluster; their kinds are pinned at the unit +level, where the analyzer/editor need only the document bytes — no controller, +no CRD. **That is exactly the point:** the pipeline never consults the +controller, only the document, so a static document is a faithful proof of the +mirror/edit behavior. + +### Live e2e pin — `test/e2e/helmrelease_mirror_edit_e2e_test.go` + +Mirrors launch use case 2 ("roll out a new version — a chart version on a Flux +`HelmRelease`"): + +1. A `WatchRule` selecting `helmreleases` → `GitTarget`; create a `HelmRelease` + live with `spec.chart.spec.version: X` (its source need not resolve — the + operator mirrors the *object*, not Flux's reconcile result). +2. Assert the operator commits the canonical file + `{path}/{ns}/helm.toolkit.fluxcd.io/helmreleases/{name}.yaml`. +3. Seed a hand-authored comment into the committed file (semantically inert, so + the operator leaves it alone). +4. Patch live `spec.chart.spec.version` → `Y`. +5. Assert the file shows `Y`, the comment survives, and `X` is gone — an + in-place, comment-preserving edit of a control-plane CR, identical to the + `ConfigMap` case ([inplace_edit_e2e_test.go](../../../test/e2e/inplace_edit_e2e_test.go)). + +### Unit corpus — `internal/git/manifestedit/testdata/corpus` + +Add `helmrelease.yaml`, `argocd-application.yaml`, `kro-podinfoapp.yaml`. The +two existing globbed gates then automatically cover them: + +- `TestCorpusRoundTrip_ByteIdentical` — every document re-encodes byte-for-byte + (comment and formatting fidelity). +- `TestConvergence_Corpus` — perturb the projection with a label, prove the next + reconcile settles to a byte-stable no-op (a field edit converges). + +Each document carries a hand-authored comment and a realistic version/revision +field so the gates exercise the properties that matter for these kinds. + +## What F7 deliberately does not do + +- **No per-kind semantics.** A `HelmRelease`'s `chartRef`, an `Application`'s + `targetRevision`, a KRO field are ordinary scalars; the operator edits the + field a human pointed it at, it does not understand chart/app semantics. +- **No chart inflation.** `helmCharts:` in a kustomization stays refused; F7 + mirrors the `HelmRelease` *document*, it never renders a chart. +- **No new controllers in e2e.** Argo and KRO stay out of the base cluster; + adding their CRDs for a live pin is future work, wanted only if the unit-level + proof ever proves insufficient. +- **No placement changes.** New-file placement for these kinds is F4's generic + path; F7 adds no kind-specific placement. + +## Test plan + +- **Unit:** three corpus documents flow through the two globbed gates + (round-trip byte-identical, convergence perturb-then-settle) with no new test + code — the gates already glob `testdata/corpus/*.yaml`. +- **e2e:** `HelmRelease` mirror + chart-version-bump edit, comment-preserving, + under the `manager` + `f7-higher-level-krm` labels. +- **User docs:** [../../installing-apps-as-krm.md](../../installing-apps-as-krm.md) + — "adding an app is adding a KRM document," with `HelmRelease`/`Application` + examples and the chart-inflation boundary. diff --git a/docs/installing-apps-as-krm.md b/docs/installing-apps-as-krm.md new file mode 100644 index 00000000..9405dc7d --- /dev/null +++ b/docs/installing-apps-as-krm.md @@ -0,0 +1,126 @@ +# Installing apps is adding a KRM document + +GitOps Reverser mirrors and edits **Kubernetes Resource Model (KRM) documents**. +It does not care whether a document is a core `ConfigMap`, an `apps/v1` +`Deployment`, or a higher-level control-plane object such as a Flux +`HelmRelease`, an Argo CD `Application`, or a [KRO](https://kro.run) resource. +They are all just documents with an `apiVersion`, `kind`, and `metadata`, and the +operator treats them identically. + +That gives you two everyday workflows without learning the repository's layout: + +- **Install an app** → apply the KRM document to the cluster. The operator writes + it to Git, in the folder your existing files already use. +- **Roll out a new version** → edit one field on the live object (a chart + version, an image tag, a revision). The operator writes that change back into + the *same* file, in place, preserving your comments and formatting. + +## What counts as "an app" + +Anything you would normally `kubectl apply`. In KRM terms, installing an app is +adding one or more documents: + +| You want to… | The KRM document you add | +|---|---| +| Run a workload directly | a core `Deployment` / `StatefulSet` (+ `Service`, `ConfigMap`) | +| Install a Helm chart via Flux | a `helm.toolkit.fluxcd.io/v2` `HelmRelease` (+ its `HelmRepository`/`OCIRepository` source) | +| Deploy via Argo CD | an `argoproj.io/v1alpha1` `Application` | +| Use a platform abstraction | a KRO instance (e.g. `kro.run/v1alpha1` `PodInfoApp`) or any CRD your platform provides | + +### Example: a Flux HelmRelease + +```yaml +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: podinfo + namespace: apps +spec: + interval: 30m + chart: + spec: + chart: podinfo + version: "6.5.0" # bump this to roll out a new version + sourceRef: + kind: HelmRepository + name: podinfo + namespace: flux-system +``` + +Apply it and the operator commits it. Later, change `version` to `6.6.0` on the +live object (`kubectl patch`, your platform, or any controller) and the operator +edits that one line in the committed file — your comment on it stays put. + +### Example: an Argo CD Application + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: guestbook + namespace: argocd +spec: + project: default + source: + repoURL: https://github.com/argoproj/argocd-example-apps.git + path: guestbook + targetRevision: "1.4.0" # bump this to roll out a new revision + destination: + server: https://kubernetes.default.svc + namespace: guestbook +``` + +## Telling the operator to watch these kinds + +A `WatchRule` selects resources by plural name and (optionally) API group. To +mirror HelmReleases in a namespace: + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: watch-helmreleases + namespace: apps +spec: + targetRef: + name: my-gittarget + rules: + - apiGroups: ["helm.toolkit.fluxcd.io"] + resources: ["helmreleases"] +``` + +The kind's CRD must be installed and served in the cluster — the operator +resolves what to watch from the live API surface, so any served kind (core, CRD, +or aggregated) is watchable. See [`configuration.md`](configuration.md) for the +full `GitProvider` / `GitTarget` / `WatchRule` model. + +## Where the document lands in Git + +A brand-new document follows the layout your folder already uses (sibling +inference), or the `GitTarget`'s declared placement policy. Absent any existing +convention, the cold-start default path is: + +```text +{GitTarget path}/{namespace}/{group}/{resource}/{name}.yaml +``` + +so the HelmRelease above lands at +`.../apps/helm.toolkit.fluxcd.io/helmreleases/podinfo.yaml`. Once a document +exists, it is always edited **in place** wherever it already lives — the operator +never moves or duplicates it. See +[`configuration.md` → Where new resources are written](configuration.md#where-new-resources-are-written-specplacement). + +## The boundary: documents, not chart inflation + +"No Helm" means the operator never *renders* a chart. It will not run +`helm template`, it will not expand a kustomization `helmCharts:` generator, and +it will not turn a chart into a tree of manifests. What it does is mirror and +edit the `HelmRelease` **document** — the small piece of intent that says "install +this chart at this version." The chart is inflated by Flux, in the cluster, as +always. + +The same line holds for the other kinds: the operator edits the fields on the +`Application` or KRO document that a human would edit; it does not understand or +reproduce what Argo CD or KRO then does with them. That boundary is what keeps +every edit round-trippable — exactly one writable destination in Git for each +change — and it is a deliberate part of the support contract, not a gap. diff --git a/internal/git/manifestedit/testdata/corpus/argocd-application.yaml b/internal/git/manifestedit/testdata/corpus/argocd-application.yaml new file mode 100644 index 00000000..6b43420f --- /dev/null +++ b/internal/git/manifestedit/testdata/corpus/argocd-application.yaml @@ -0,0 +1,18 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: guestbook + namespace: argocd +spec: + project: default + source: + repoURL: https://github.com/argoproj/argocd-example-apps.git + path: guestbook + targetRevision: "1.4.0" # bump to roll out a new revision + destination: + server: https://kubernetes.default.svc + namespace: guestbook + syncPolicy: + automated: + prune: true + selfHeal: true diff --git a/internal/git/manifestedit/testdata/corpus/helmrelease.yaml b/internal/git/manifestedit/testdata/corpus/helmrelease.yaml new file mode 100644 index 00000000..65fb0267 --- /dev/null +++ b/internal/git/manifestedit/testdata/corpus/helmrelease.yaml @@ -0,0 +1,21 @@ +# podinfo release — bump spec.chart.spec.version to roll out a new version +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: podinfo + namespace: apps +spec: + interval: 30m + chart: + spec: + chart: podinfo + version: "6.5.0" # pinned chart version + sourceRef: + kind: HelmRepository + name: podinfo + namespace: flux-system + install: + remediation: + retries: 3 + values: + replicaCount: 2 diff --git a/internal/git/manifestedit/testdata/corpus/kro-podinfoapp.yaml b/internal/git/manifestedit/testdata/corpus/kro-podinfoapp.yaml new file mode 100644 index 00000000..e6e92b9b --- /dev/null +++ b/internal/git/manifestedit/testdata/corpus/kro-podinfoapp.yaml @@ -0,0 +1,11 @@ +# a KRO-generated instance (schema comes from a ResourceGraphDefinition) +apiVersion: kro.run/v1alpha1 +kind: PodInfoApp +metadata: + name: frontend + namespace: apps +spec: + replicas: 1 + message: hello from kro + color: "#34577c" # brand color + logoURL: https://demo.configbutler.ai/podinfo-assets/frontend.jpg diff --git a/test/e2e/helmrelease_mirror_edit_e2e_test.go b/test/e2e/helmrelease_mirror_edit_e2e_test.go new file mode 100644 index 00000000..00012f6e --- /dev/null +++ b/test/e2e/helmrelease_mirror_edit_e2e_test.go @@ -0,0 +1,176 @@ +// SPDX-License-Identifier: Apache-2.0 + +package e2e + +import ( + "fmt" + "os" + "path/filepath" + "strings" + "time" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +// F7: higher-level KRM objects are first-class documents. This spec proves the +// mirror/edit pipeline treats a Flux HelmRelease — a control-plane custom +// resource, the canonical "install an app" object — exactly like a core +// resource: it is mirrored to Git on create, and a live field edit (a chart +// version bump, launch use case 2) round-trips into the mirrored file in place, +// preserving hand-authored formatting. The generic-CRD case is already pinned by +// crd_lifecycle_e2e_test.go; this pins a real, named higher-level type. +// See docs/design/gitops-api/f7-higher-level-krm-documents.md. +var _ = Describe("Manager F7 Higher-Level KRM (HelmRelease)", + Label("manager", "f7-higher-level-krm"), Ordered, func() { + var ( + testNs string + repo *RepoArtifacts + providerName = "f7-helmrelease-provider" + destName = "f7-helmrelease-dest" + ruleName = "f7-helmrelease-rule" + hrName = "podinfo" + gitPath = "e2e/f7-helmrelease" + ) + + const ( + initialVersion = "6.5.0" + bumpedVersion = "6.6.0" + preservedComment = "# gitops-reverser-e2e: preserve-this-helmrelease-comment" + repoName = "e2e-f7-helmrelease" + ) + + // helmReleaseRepoPath is the canonical mirror path for a grouped custom + // resource: ////.yaml. + helmReleaseRepoPath := func() string { + return filepath.Join(gitPath, testNs, "helm.toolkit.fluxcd.io", "helmreleases", hrName+".yaml") + } + + BeforeAll(func() { + By("creating the f7-helmrelease test namespace") + testNs = testNamespaceFor("manager-f7-helmrelease") + _, _ = kubectlRun("create", "namespace", testNs) + + By("setting up Gitea repo and credentials") + repo = SetupRepo(resolveE2EContext(), testNs, fmt.Sprintf("%s-%d", repoName, GinkgoRandomSeed())) + + _, err := kubectlRunInNamespace(testNs, "apply", "-f", repo.SecretsYAML) + Expect(err).NotTo(HaveOccurred(), "failed to apply git secrets to test namespace") + + applySOPSAgeKeyToNamespace(testNs) + + By("creating the GitProvider") + createGitProviderWithURLInNamespace(providerName, testNs, repo.GitSecretHTTP, repo.RepoURLHTTP) + verifyResourceStatus("gitprovider", providerName, testNs, "True", "Ready", "") + }) + + AfterAll(func() { + _, _ = kubectlRunInNamespace(testNs, "delete", "helmrelease", hrName, "--ignore-not-found=true") + cleanupWatchRule(ruleName, testNs) + cleanupGitTarget(destName, testNs) + _, _ = kubectlRunInNamespace(testNs, "delete", "gitprovider", providerName, "--ignore-not-found=true") + cleanupNamespace(testNs) + }) + + It("mirrors a HelmRelease and round-trips a chart-version bump in place", func() { + By("creating the GitTarget and a HelmRelease WatchRule") + createGitTarget(destName, testNs, providerName, gitPath, "main") + err := applyFromTemplate("test/e2e/templates/manager/watchrule-helmrelease.tmpl", struct { + Name string + Namespace string + DestinationName string + }{Name: ruleName, Namespace: testNs, DestinationName: destName}, testNs) + Expect(err).NotTo(HaveOccurred(), "failed to apply HelmRelease WatchRule") + verifyResourceCondition("gittarget", destName, testNs, "Validated", "True", "OK", "") + verifyResourceStatus("watchrule", ruleName, testNs, "True", "Ready", "") + waitForStreamsRunning(destName, testNs) + + By(fmt.Sprintf("creating the HelmRelease with chart version %s", initialVersion)) + err = applyFromTemplate("test/e2e/templates/manager/helmrelease.tmpl", struct { + Name string + Namespace string + Version string + }{Name: hrName, Namespace: testNs, Version: initialVersion}, testNs) + Expect(err).NotTo(HaveOccurred(), "failed to apply HelmRelease") + + By("waiting for the operator to mirror the HelmRelease to its canonical path") + fullPath := filepath.Join(repo.CheckoutDir, helmReleaseRepoPath()) + Eventually(func(g Gomega) { + pullLatestRepoState(g, repo.CheckoutDir) + content, readErr := os.ReadFile(fullPath) + g.Expect(readErr).NotTo(HaveOccurred(), "HelmRelease file must exist at %s", fullPath) + body := string(content) + g.Expect(body).To(ContainSubstring("kind: HelmRelease")) + g.Expect(body).To(ContainSubstring(initialVersion), "the initial chart version must be mirrored") + g.Expect(body).NotTo(ContainSubstring("status:"), "status must never be committed") + }, 90*time.Second, 3*time.Second).Should(Succeed()) + + By("seeding a hand-authored comment into the committed file") + seedHeadCommentIntoRepoFile(repo, testNs, helmReleaseRepoPath(), preservedComment) + Eventually(func(g Gomega) { + pullLatestRepoState(g, repo.CheckoutDir) + content, readErr := os.ReadFile(fullPath) + g.Expect(readErr).NotTo(HaveOccurred()) + g.Expect(string(content)).To(ContainSubstring(preservedComment)) + }, 60*time.Second, 2*time.Second).Should(Succeed()) + + By(fmt.Sprintf("bumping the live chart version to %s to trigger an in-place edit", bumpedVersion)) + _, err = kubectlRunInNamespace(testNs, "patch", "helmrelease", hrName, + "--type=merge", "--patch", + fmt.Sprintf(`{"spec":{"chart":{"spec":{"version":"%s"}}}}`, bumpedVersion)) + Expect(err).NotTo(HaveOccurred(), "HelmRelease patch should succeed") + + By("verifying the edit landed in place: new version written, comment preserved, old version gone") + Eventually(func(g Gomega) { + pullLatestRepoState(g, repo.CheckoutDir) + content, readErr := os.ReadFile(fullPath) + g.Expect(readErr).NotTo(HaveOccurred()) + body := string(content) + g.Expect(body).To(ContainSubstring(bumpedVersion), "the bumped chart version must be written") + g.Expect(body).To(ContainSubstring(preservedComment), + "the hand-authored comment must survive the in-place edit") + g.Expect(body).NotTo(ContainSubstring(initialVersion)) + }, 90*time.Second, 3*time.Second).Should(Succeed()) + + By("✅ HelmRelease mirrored and edited in place like any KRM document") + }) + }) + +// seedHeadCommentIntoRepoFile prepends a hand-authored comment as the document +// head comment of the mirrored file and pushes it to main, authenticating the +// local checkout's origin from the GitTarget's Git Secret. It retries once over +// a remote race by rebasing on origin/main. A document head comment is the most +// layout-agnostic anchor: it survives an in-place field edit because the editor +// only rewrites changed nodes. +func seedHeadCommentIntoRepoFile(repo *RepoArtifacts, namespace, relPath, comment string) { + GinkgoHelper() + + configureRepoOriginWithCredentials(repo, namespace) + + mustGit := func(args ...string) { + out, gitErr := gitRun(repo.CheckoutDir, args...) + Expect(gitErr).NotTo(HaveOccurred(), fmt.Sprintf("git %s: %s", strings.Join(args, " "), out)) + } + + if _, err := gitRun(repo.CheckoutDir, "fetch", "origin", "main"); err == nil { + mustGit("checkout", "-B", "main", "origin/main") + mustGit("reset", "--hard", "origin/main") + } else { + mustGit("checkout", "--orphan", "main") + _, _ = gitRun(repo.CheckoutDir, "rm", "-rf", ".") + } + + full := filepath.Join(repo.CheckoutDir, relPath) + content, readErr := os.ReadFile(full) + Expect(readErr).NotTo(HaveOccurred(), "committed file must exist before seeding a comment") + seeded := comment + "\n" + string(content) + Expect(os.WriteFile(full, []byte(seeded), 0o600)).To(Succeed()) + + mustGit("add", relPath) + mustGit("commit", "-m", "e2e: seed hand-authored comment on HelmRelease") + if _, pushErr := gitRun(repo.CheckoutDir, "push", "origin", "HEAD:main"); pushErr != nil { + mustGit("fetch", "origin", "main") + mustGit("rebase", "origin/main") + mustGit("push", "origin", "HEAD:main") + } +} diff --git a/test/e2e/templates/manager/helmrelease.tmpl b/test/e2e/templates/manager/helmrelease.tmpl new file mode 100644 index 00000000..015b9eb9 --- /dev/null +++ b/test/e2e/templates/manager/helmrelease.tmpl @@ -0,0 +1,19 @@ +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: {{ .Name }} + namespace: {{ .Namespace }} +spec: + # Suspended so the helm-controller leaves the object inert: we are proving the + # mirror/edit pipeline treats a higher-level KRM document like any other, not + # exercising Flux itself. The referenced source need not exist. + suspend: true + interval: 30m + chart: + spec: + chart: podinfo + version: "{{ .Version }}" + sourceRef: + kind: HelmRepository + name: podinfo + namespace: {{ .Namespace }} diff --git a/test/e2e/templates/manager/watchrule-helmrelease.tmpl b/test/e2e/templates/manager/watchrule-helmrelease.tmpl new file mode 100644 index 00000000..c9fdf343 --- /dev/null +++ b/test/e2e/templates/manager/watchrule-helmrelease.tmpl @@ -0,0 +1,11 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: {{ .Name }} + namespace: {{ .Namespace }} +spec: + targetRef: + name: {{ .DestinationName }} + rules: + - apiGroups: ["helm.toolkit.fluxcd.io"] + resources: ["helmreleases"] From bfa6e306df85375334ba9e52647e6103c681340f Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 7 Jul 2026 08:50:48 +0000 Subject: [PATCH 2/2] docs(gitops-api): stress capturing intent, not rendered output The install-apps guide explained that the operator won't *render* charts, but not the bigger configuration discipline: users must deliberately scope capture to authored intent (HelmRelease/Application/KRO documents) and keep the objects those controllers render (Deployments, Services, ConfigMaps, ReplicaSets, Pods) out of Git. Add a "capture intent, not the rendered output" section that explains why mirroring rendered output breaks round-trippability (controller-owned objects have no single writable destination), that the operator cannot tell an authored Deployment from a rendered one (only a few purely-runtime kinds are excluded by default), and how to draw the line: select intent kinds not workload kinds, separate intent from runtime by namespace, and mind overlaps. Frame it as the same principle as the chart-inflation boundary, applied at runtime. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/installing-apps-as-krm.md | 61 ++++++++++++++++++++++++++++++++-- 1 file changed, 59 insertions(+), 2 deletions(-) diff --git a/docs/installing-apps-as-krm.md b/docs/installing-apps-as-krm.md index 9405dc7d..0bb6d11f 100644 --- a/docs/installing-apps-as-krm.md +++ b/docs/installing-apps-as-krm.md @@ -70,10 +70,67 @@ spec: namespace: guestbook ``` +## The design decision: capture intent, not the rendered output + +This is the part worth slowing down for. Before you point GitOps Reverser at a +cluster, decide **where and how your intent is expressed**, and configure the +operator to mirror *that* — not the objects your controllers render from it. + +Every kind in the table above is a piece of **intent** that a controller expands +into many **derived** objects. One `HelmRelease` becomes Deployments, Services, +ConfigMaps, ReplicaSets, and Pods. One Argo CD `Application` becomes whatever it +syncs. One KRO instance becomes its entire resource graph. You want Git to hold +the small authored document, not the large derived tree — the derived tree is a +*projection* the controller recomputes on every reconcile. + +**Why mirroring the output is wrong**, not just noisy: + +- It breaks the [governing rule](design/gitops-api/README.md) — round-trippability. + A rendered object is *owned by a controller*: it carries generated names, hash + suffixes, injected defaults, and controller-managed metadata. There is no single + writable human destination for it, so an edit can't round-trip. Capturing + intent keeps exactly one writable destination per change. +- It churns. Reconcilers rewrite their outputs constantly; a repo full of + rendered Deployments and ReplicaSets commits on every loop. +- It duplicates state that already exists as intent — the `HelmRelease` *is* the + Deployment, expressed once, at the level a human actually edits. + +This is the same principle as the [chart-inflation boundary](#the-boundary-documents-not-chart-inflation) +below, applied at runtime: don't put *derived* state in Git, at any level. + +**The catch: the operator cannot tell intent from output for you.** A +`Deployment` is a `Deployment` whether you hand-authored it or Flux rendered it. +GitOps Reverser ignores a few purely-runtime kinds by default — Pods, Events, +Endpoints/EndpointSlices, Leases, ControllerRevisions, Jobs, CronJobs — but it +does **not** ignore Deployments, ReplicaSets, Services, or ConfigMaps, because +those are often exactly the intent you *do* want to capture. Drawing the line is +your job, and it is worth doing deliberately. + +**How to draw it:** + +1. **Select intent kinds, not workload kinds.** In a namespace where a controller + renders output, watch `helmreleases` / `applications` / your KRO CRD — not the + `deployments` and `services` they produce. Avoid a catch-all `resources: ["*"]` + there. +2. **Separate intent from runtime, ideally by namespace.** The cleanest, most + robust setup keeps authored intent in its own namespace (or cluster area) that + your `GitTarget`/`WatchRule` points at, and lets the rendered workloads live + elsewhere, unwatched. Then "what to capture" is answered by *where*, not by a + fragile per-kind allow/deny list. +3. **Mind overlaps.** If a controller renders into the same namespace as other + authored intent, keep the rule kind-scoped (watch only the intent kinds), so + you never recapture the rendered output alongside it. + +Getting this balance right up front is most of the work of adopting the tool. If +you skip it, the symptom is unmistakable: the repository fills with +controller-owned objects that change on every reconcile and that you cannot +meaningfully edit — noise, not intent. + ## Telling the operator to watch these kinds -A `WatchRule` selects resources by plural name and (optionally) API group. To -mirror HelmReleases in a namespace: +A `WatchRule` selects resources by plural name and (optionally) API group. Keep +it **scoped to intent kinds** (see the section above). To mirror HelmReleases in +a namespace — and nothing they render: ```yaml apiVersion: configbutler.ai/v1alpha3