Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .coverage-baseline
Original file line number Diff line number Diff line change
@@ -1 +1 @@
74.5
75.2
11 changes: 2 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,6 @@ simpler for other teams.
> managed control planes (EKS/GKE/AKS) generally do not expose. Without it the operator still mirrors
> state, with commits authored by the configured committer. **Valkey/Redis is required either way** — it
> holds each GitTarget's watch resume state so work is re-picked up after a restart or reconnect.
>
> If exact per-user authorship matters but you do not want to own kube-apiserver audit delivery yourself,
> I welcome conversations about a managed ConfigButler path.

## How it works

Expand Down Expand Up @@ -122,12 +119,8 @@ Directions we may revisit later live in [docs/TODO.md](docs/TODO.md) and [docs/f
## Quick start

This quick start sets up **committer-only mode**: the operator mirrors watched Kubernetes state into Git,
and commits are authored by the configured committer identity. That is the easiest way to prove the
workflow works.

Valkey/Redis is still required: it holds each GitTarget's watch resume state so the operator re-picks up
where it left off after a restart or reconnect. Named commit authors can be added later with
kube-apiserver audit delivery.
commits are authored by the configured committer identity, and named authors can be added later. It is the
easiest way to prove the workflow works.

**Prerequisites**

Expand Down
42 changes: 42 additions & 0 deletions api/v1alpha3/gittarget_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,48 @@ type GitTargetSpec struct {
// Encryption defines encryption settings for Secret resource writes.
// +optional
Encryption *EncryptionSpec `json:"encryption,omitempty"`

// Placement declares where NEW resources are written. It has no effect on a
// resource that already has a document in Git — that document is always
// updated in place at its existing location, wherever that is. Mutable: a
// change only affects resources created after the change.
// +optional
Placement *GitTargetPlacementSpec `json:"placement,omitempty"`
}

// GitTargetPlacementSpec declares where NEW resources are written when no document
// for their identity exists yet in Git — one exact-type map plus a fallback
// default template (Option B2 of
// docs/design/manifest/version2/gittarget-new-file-placement-rules.md). There is
// deliberately no separate "sensitive" placement block: sensitivity is a
// write-safety classification the controller owns (encrypt the content, keep the
// path identity-complete, never append or co-mingle), not a second placement
// namespace the user has to configure. A user routes Secrets the same way they
// route anything else — by naming their type in ByType. When a resource's type
// has no ByType entry and no Default, placement falls back to following the layout
// already established by sibling resources in the repository, and finally to the
// canonical, versionless {namespaceOrCluster}/{group}/{resource}/{name}.yaml path
// when there is nothing to follow. Because that fallback omits the API version,
// objects that differ only by version share a file; a target that watches several
// versions of the same group/resource and wants them separated must use a
// ByType/Default template that includes {version}.
type GitTargetPlacementSpec struct {
// ByType maps an exact resource type key ("{group}/{version}/{resource}", e.g.
// "v1/configmaps", "apps/v1/deployments", or "v1/secrets"; core resources omit
// the group) to the path template used for a new resource of that type. A path
// selected for a sensitive resource (Secrets, plus any operator-configured
// sensitive type) must be identity-complete so it cannot collide two distinct
// sensitive resources onto one file.
// +optional
ByType map[string]string `json:"byType,omitempty"`

// Default is the path template used for a new resource whose type has no ByType
// entry. Omitted, it falls through to sibling-layout inference and then the
// built-in canonical path. A bundling default (one that is not identity-complete,
// such as "all.yaml") is only valid when a sensitive resource can never reach it
// — give every sensitive type an explicit identity-complete ByType entry.
// +optional
Default string `json:"default,omitempty"`
}

// GitTargetStatus defines the observed state of GitTarget.
Expand Down
5 changes: 5 additions & 0 deletions api/v1alpha3/watchrule_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,11 @@ type ResourceRule struct {
// - ["v1", "v1beta1"] matches both versions
// - ["*"] matches all served versions
// - [] matches the preferred served version
//
// Multi-version note: the built-in cold-start Git path is versionless, so two
// objects that differ only by API version resolve to the same file. To watch
// several versions of a group/resource and keep them in separate files, give the
// GitTarget a placement template that includes {version} (see GitTargetPlacementSpec).
// +optional
APIVersions []string `json:"apiVersions,omitempty"`

Expand Down
27 changes: 27 additions & 0 deletions api/v1alpha3/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 27 additions & 0 deletions config/crd/bases/configbutler.ai_gittargets.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,33 @@ spec:
Immutable: delete and recreate the GitTarget to change its destination.
minLength: 1
type: string
placement:
description: |-
Placement declares where NEW resources are written. It has no effect on a
resource that already has a document in Git — that document is always
updated in place at its existing location, wherever that is. Mutable: a
change only affects resources created after the change.
properties:
byType:
additionalProperties:
type: string
description: |-
ByType maps an exact resource type key ("{group}/{version}/{resource}", e.g.
"v1/configmaps", "apps/v1/deployments", or "v1/secrets"; core resources omit
the group) to the path template used for a new resource of that type. A path
selected for a sensitive resource (Secrets, plus any operator-configured
sensitive type) must be identity-complete so it cannot collide two distinct
sensitive resources onto one file.
type: object
default:
description: |-
Default is the path template used for a new resource whose type has no ByType
entry. Omitted, it falls through to sibling-layout inference and then the
built-in canonical path. A bundling default (one that is not identity-complete,
such as "all.yaml") is only valid when a sensitive resource can never reach it
— give every sensitive type an explicit identity-complete ByType entry.
type: string
type: object
providerRef:
description: |-
ProviderRef references the GitProvider that backs this target.
Expand Down
5 changes: 5 additions & 0 deletions config/crd/bases/configbutler.ai_watchrules.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,11 @@ spec:
- ["v1", "v1beta1"] matches both versions
- ["*"] matches all served versions
- [] matches the preferred served version

Multi-version note: the built-in cold-start Git path is versionless, so two
objects that differ only by API version resolve to the same file. To watch
several versions of a group/resource and keep them in separate files, give the
GitTarget a placement template that includes {version} (see GitTargetPlacementSpec).
items:
type: string
type: array
Expand Down
61 changes: 43 additions & 18 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -422,24 +422,26 @@ local SHA from before the retry.

## What It Writes to Git

A `GitTarget` owns one subtree (`spec.path`) on one branch. New objects are placed at a canonical REST
like path; once a document exists it is edited in place wherever it already lives. A populated target
looks like:
A `GitTarget` owns one subtree (`spec.path`) on one branch. A new object follows the layout the folder
already has (or a declared policy); once a document exists it is edited in place wherever it already
lives. A populated target seeded from empty looks like:

```text
team-a-config/ # GitTarget spec.path
├── README.md # operator-managed bootstrap file
├── .sops.yaml # present only when encryption is configured
├── v1/
│ ├── configmaps/team-a/app-config.yaml
│ └── secrets/team-a/db-creds.sops.yaml # sensitive types are SOPS/age encrypted
└── apps/
└── v1/deployments/team-a/api.yaml
├── team-a/
│ ├── configmaps/app-config.yaml
│ ├── secrets/db-creds.sops.yaml # sensitive types are SOPS/age encrypted
│ └── apps/deployments/api.yaml
```

The path shape is `{spec.path}/{group}/{version}/{resource}/{namespace}/{name}.yaml` (the empty core
group is omitted, sensitive resources get a `.sops.yaml` suffix). Details and the placement policy are in
[Git Write Architecture](#git-write-architecture).
The **built-in default** path is `{spec.path}/{namespace}/{group}/{resource}/{name}.yaml` — namespace
first, the API group omitted for core resources, no version segment, and a `.sops.yaml` suffix for
sensitive resources; a cluster-scoped resource uses the literal `cluster/` in place of the namespace.
But that default is only the cold-start seed: a new resource first follows its **siblings'** existing
layout, and a `GitTarget` can declare its own placement policy. Details and the placement policy are in
[File Placement](#file-placement).

***

Expand Down Expand Up @@ -901,7 +903,8 @@ hydrates only touched files into buffers for the commit, and flushes only change

* **Upserts:** if a managed document for the resource already exists, patch it in place (preserving
siblings in a multi document file); if it is sensitive, encrypt the whole document again at its existing
path; if no document exists, create a new file at the canonical placement path.
path; if no document exists, place a new file per [File Placement](#file-placement) (declared policy,
then sibling inference, then the canonical default).
* **Kustomize override edit-through:** a live value produced by a well-formed `images:` or `replicas:`
entry in the document's kustomization chain is written back to that entry (comment-preserving, only
fields the entry already declares); the source manifest keeps its bytes. Anything the inversion cannot
Expand All @@ -915,11 +918,31 @@ hydrates only touched files into buffers for the commit, and flushes only change

### File Placement

New resources use the canonical REST like path
`{spec.path}/{group}/{version}/{resource}/{namespace}/{name}.yaml`. The core group's empty segment is
omitted (`{spec.path}/v1/configmaps/ns/name.yaml`), and sensitive resources use a `.sops.yaml` suffix.
Existing resources are **match first**: once a document exists in Git, updates and deletes use its current
location instead of recomputing placement.
Placement runs **only for a resource with no existing document** in the target. Existing resources are
**match first**: once a document exists in Git, updates and deletes use its current location — found by
manifest identity, not by path — instead of recomputing placement. So a change to how new files are
placed never moves a file already in Git. A new resource is placed by the first of these that applies
([internal/manifestanalyzer/placement.go](../internal/manifestanalyzer/placement.go),
[design](design/manifest/version2/gittarget-new-file-placement-rules.md)):

1. **Declared policy (`spec.placement`).** A `GitTarget` can declare a `byType` map (exact
`[group/]version/resource` → path template) plus a `default` template, rendered from a small
brace-variable path language (`{namespace}`, `{group}`, `{resource}`, `{name}`, …).
2. **Sibling inference.** With no matching declared template, the new resource follows the layout its
siblings already use — appended to the bundle its type shares, or placed one-per-file beside them —
so pointing a target at an existing folder just continues that folder's convention. When the whole
subtree is governed by one supported kustomization and the type is brand new, the file lands beside
that kustomization and gets a `resources:` entry.
3. **Canonical fallback.** With nothing to follow (an empty repo, a brand-new type), the built-in default
`{spec.path}/{namespace}/{group}/{resource}/{name}.yaml` — namespace-first, group omitted for core, no
version, `_cluster/` for cluster-scoped, `.sops.yaml` for sensitive — so a fresh target is deterministic
and self-propagating.

Sensitivity is a write-safety classifier, not a placement input: whatever path is chosen, a sensitive
resource is written encrypted, is never appended to an existing file, and is never co-mingled with a
plaintext document. When those guarantees cannot be honoured (e.g. a bundling `default` would route a
sensitive resource into a shared file), the resource is **skipped fail-safe** — logged per-resource and
counted in the resync summary (`placementSkipped`) — rather than written unsafely.

### Bootstrap, Encryption, and Signing

Expand Down Expand Up @@ -1062,7 +1085,9 @@ Current limitations:
* **No multi cluster routing;** cluster ID path segments on `/audit-webhook` are rejected.
* **`deletecollection`** is reconciled by the watch (each item arrives as its own `DELETED`, or the
mark-and-sweep reconciles them on replay).
* **New file placement** is the canonical path, not user configurable.
* **A fail-safe placement skip has no dedicated status condition.** A resource the writer refuses to
place unsafely is logged per-resource and counted in the resync summary (`placementSkipped`), but is
not (yet) surfaced as a distinct GitTarget status condition.

***

Expand Down
Loading