From abcefc3212c5c2297caee86456db1461cb1afb9d Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Wed, 27 May 2026 19:54:44 +0000 Subject: [PATCH 01/16] Add cosa-basics AI agent skill for COSA workflows Add an OpenCode agent skill that guides AI coding agents through the core coreos-assembler workflow: initializing a build directory, building from source or fetching published builds, and launching local QEMU VMs. The skill acts as a workflow guide rather than duplicating documentation. It references the canonical docs (docs/building-fcos.md, docs/cosa/run.md, src/cmd-init, src/cmd-buildfetch, etc.) and instructs the agent to read them at runtime for command details. This avoids staleness from having multiple sources of truth. Key workflow covered: - Step 1: Set up a COSA workdir (cosa init) - Step 2a: Build from source (cosa build + cosa osbuild qemu) - Step 2b: Fetch a published build (cosa buildfetch), defaulting to --artifact qemu --decompress for local QEMU usage - Step 3: Launch a VM (cosa run) Both container mode (default, via the cosa() bash alias) and local development mode are documented. Cloud platform workflows are explicitly out of scope for a future cosa-cloud skill. Assisted-by: --- .agents/skills/cosa-basics/SKILL.md | 171 ++++++++++++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 .agents/skills/cosa-basics/SKILL.md diff --git a/.agents/skills/cosa-basics/SKILL.md b/.agents/skills/cosa-basics/SKILL.md new file mode 100644 index 0000000000..bc95a2333f --- /dev/null +++ b/.agents/skills/cosa-basics/SKILL.md @@ -0,0 +1,171 @@ +--- +name: cosa-basics +description: Initialize a COSA workdir, build or fetch Fedora CoreOS images, and launch local QEMU VMs using coreos-assembler +--- + +# COSA Basics + +This skill guides you through the core coreos-assembler (COSA) workflow: +initializing a build directory, obtaining a CoreOS build (either from source +or by fetching a published build), and launching a local QEMU VM. + +## Out of scope + +Cloud platform image building and uploading (AWS, GCP, Azure, etc.) are not +covered here. Use a dedicated cosa-cloud skill for those workflows. + +## Key documentation references + +Do NOT duplicate these docs. Read them at runtime for command details: + +| File | What it covers | +|------|----------------| +| `docs/building-fcos.md` | Full build walkthrough, prerequisites, the `cosa()` bash alias | +| `docs/cosa.md` | Command reference overview (all cosa subcommands) | +| `docs/cosa/run.md` | Detailed `cosa run` and QEMU VM options | +| `docs/working.md` | Overrides, customization, advanced usage | +| `docs/devel.md` | Running cosa locally outside a container | +| `src/cmd-init` | `cosa init` source -- all supported flags | +| `src/cmd-buildfetch` | `cosa buildfetch` source -- all supported flags | + +## Running COSA + +### Container mode (default) + +COSA is designed to run inside a container. The container image is: + +``` +quay.io/coreos-assembler/coreos-assembler +``` + +Before running any cosa commands, read `docs/building-fcos.md` and extract the +`cosa()` bash alias function defined there. This alias wraps `podman run` with +the necessary privileges, device mounts (`/dev/kvm`, `/dev/fuse`), user +namespace mapping, and volume mounts. The current working directory is mounted +as `/srv/` inside the container. + +Set up the alias in the user's shell before running any cosa commands. + +### Local mode (for cosa developers) + +Developers who have cosa installed locally can run commands directly without +the container wrapper. Read `docs/devel.md` for details on this workflow. The +commands themselves are the same; only the execution environment differs. + +## Workflow + +### Step 1: Set up a COSA workdir + +1. Ask the user for a working directory path. Default to `./fcos` if they + don't specify one. +2. If the directory does not exist, create it. +3. `cd` into the directory. +4. Set up the `cosa()` bash alias (read it from `docs/building-fcos.md`). +5. Initialize the workdir: + +``` +cosa init https://github.com/coreos/fedora-coreos-config +``` + +The user can override the config repo URL. Additional useful flags (read +`src/cmd-init` for the full list): + +- `--branch BRANCH` -- use a specific branch of the config repo +- `--variant VARIANT` -- select a build variant +- `--force` -- allow init in a non-empty directory + +After init, the directory structure will contain `src/config/` (the cloned +config repo), plus `builds/`, `cache/`, `tmp/`, and `overrides/` directories +that are populated during builds. + +### Step 2: Obtain a build + +Choose one of the following paths based on what the user wants to do. + +#### Option A: Build from source + +This builds a fresh CoreOS image from the config repo and RPM sources: + +``` +cosa build +cosa osbuild qemu +``` + +`cosa build` creates the bootable OCI container (ostree commit). `cosa osbuild +qemu` derives a QEMU disk image from that container. Both are required before +`cosa run` will work. + +Note: `cosa fetch` is a no-op in the current build flow. Package fetching +happens automatically inside `cosa build` via buildah. + +#### Option B: Fetch an existing published build + +This downloads a previously published build from the Fedora CoreOS build +servers. Use this when you want to test or inspect an existing release without +building from source. + +**Default flags:** Always include `--artifact qemu --decompress` unless the +user specifically asks for a different platform artifact. The published +artifacts are stored compressed on the server; `--decompress` expands them so +`cosa run` can use the resulting qcow2 directly. + +**Fetch the latest build for a stream:** + +``` +cosa buildfetch --stream --artifact qemu --decompress +``` + +Available streams: `stable`, `testing`, `testing-devel`, `next`, `rawhide` + +If the user does not specify a stream, default to `stable`. + +**Fetch a specific build by ID:** + +``` +cosa buildfetch --stream -b --artifact qemu --decompress +``` + +The stream can be auto-detected from the build ID's version number (the third +dotted component maps to a stream), but providing `--stream` explicitly is +safer. Read `src/cmd-buildfetch` for the version-to-stream mapping. + +**If the user wants a non-qemu artifact:** replace `--artifact qemu` with the +appropriate artifact name (e.g., `metal`, `aws`, `gcp`). But note that cloud +platform workflows are better served by a dedicated cosa-cloud skill. + +### Step 3: Launch the VM + +``` +cosa run +``` + +This launches a QEMU VM using the latest build image with `-snapshot` (changes +are discarded on exit). By default it establishes an SSH connection with +auto-login as the `core` user. + +**Common options** (read `docs/cosa/run.md` for the full list): + +| Flag | Purpose | +|------|---------| +| `-c` / `--devshell-console` | Use serial console instead of SSH (see full boot, GRUB menu) | +| `--qemu-image PATH` | Use a specific qcow2 image instead of the latest build | +| `-B` / `--butane PATH` | Pass a Butane config for the VM | +| `-i` / `--ignition PATH` | Pass an Ignition config for the VM | +| `--add-disk SIZE[:OPTS]` | Attach additional disks | +| `--kargs 'ARGS'` | Append kernel arguments | +| `-m SIZE` | Set VM memory in MB | + +**Exiting the VM:** + +- SSH mode: `exit` or `Ctrl-D` +- Console mode: `Ctrl-a x` + +## Important notes + +- Each `cosa` command (via the alias) creates a transient container. Use + `cosa shell` for a persistent interactive session inside the container. +- `/dev/kvm` is required for both building disk images and running VMs. The + host must be bare metal or have nested virtualization enabled. +- The build directory at `$PWD` is mounted at `/srv/` inside the container. + Build artifacts persist across container invocations. +- The workdir's `builds/latest` symlink always points to the most recent build. From 9ea05088ea243b384f0758f9c7028467cb074108 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Thu, 28 May 2026 16:31:07 -0400 Subject: [PATCH 02/16] docs/cosa/run: add more info about --butane/--ignition opts Just adds some clarity to the options and how they behave. Assisted-by: --- docs/cosa/run.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/cosa/run.md b/docs/cosa/run.md index f17c253248..7358600dba 100644 --- a/docs/cosa/run.md +++ b/docs/cosa/run.md @@ -21,7 +21,22 @@ sections discuss a few in more details. Using `cosa run` is a very effective way to iterate on your Butane or Ignition config. Use the `--butane/-B` or `--ignition/-i` switches respectively to pass -the config path. +the config path: + +``` +$ cosa run -B my-config.bu +$ cosa run -i my-config.ign +``` + +A few things to keep in mind: + +- `--butane` and `--ignition` are **mutually exclusive** — you can pass one or + the other, but not both. +- Butane configs are **automatically transpiled** to Ignition, so there is no + need to run `butane` manually before passing the file. +- Your config becomes the **base** Ignition config for the VM. COSA + automatically merges its own built-in fragments on top (e.g., autologin when + using devshell mode), so your config is never replaced. ## Using the serial console From 7e959c571e1f40665afc13b6c1b650210d0e1d02 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Thu, 28 May 2026 17:06:07 -0400 Subject: [PATCH 03/16] docs: remove mentions of `testiso` We dropped testiso as a separate command in https://github.com/anomalyco/opencode/pull/23794 so let's update the docs. Assisted-by: --- docs/cosa/run.md | 4 ++-- docs/kola.md | 23 +++++++++++------------ 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/docs/cosa/run.md b/docs/cosa/run.md index 7358600dba..af9557b353 100644 --- a/docs/cosa/run.md +++ b/docs/cosa/run.md @@ -161,7 +161,7 @@ installed system automatically just as the live environment itself was set up.) Of course, one can also use an Ignition config or [a customized ISO](https://coreos.github.io/coreos-installer/cmd/iso/#coreos-installer-iso-customize) or the `coreos.inst.*` kargs using `--kargs` to also manually test automated -flows. (Many of these flows are covered by our `kola testiso` tests.) +flows. (Many of these flows are covered by kola's ISO install tests.) ## Multipath @@ -229,7 +229,7 @@ TARGET SOURCE FSTYPE OPTIONS (The `--ignition /run/ignition.json` is a trick for getting auto-login on the installed system automatically just as the live environment itself was set up.) -This is equivalent to our `kola testiso` multipath tests. +This is equivalent to kola's ISO multipath install tests. ## Netbooting diff --git a/docs/kola.md b/docs/kola.md index d0f683b2c0..78f71a59b5 100644 --- a/docs/kola.md +++ b/docs/kola.md @@ -207,20 +207,19 @@ After you run the kola test, you can find more information in `tmp/kola/ Date: Thu, 28 May 2026 17:07:49 -0400 Subject: [PATCH 04/16] docs/kola: flesh out the docs with additional context Added more context here, including more of the options that `kola supports`. Assisted-by: --- docs/kola.md | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/docs/kola.md b/docs/kola.md index 78f71a59b5..69d3bd7656 100644 --- a/docs/kola.md +++ b/docs/kola.md @@ -31,6 +31,46 @@ By default, kola uses the `qemu` platform with the most recently built image 1. TOC {:toc} +## Running kola via cosa + +The recommended way to run kola is through the `cosa kola` wrapper. It handles +several defaults automatically: + +- **Platform:** defaults to `-p qemu` (when not running as root and no + `-p`/`--platform` is given). +- **Output directory:** defaults to `tmp/kola` (or `$ARTIFACT_DIR/kola` if + `$ARTIFACT_DIR` is set). +- **Subcommand:** defaults to `run` if no subcommand is given. For upgrade + tests, use `cosa kola --upgrades`. +- **Build:** uses the latest build in the cosa workdir by default. Use + `--build` to select a specific build. + +For example, `cosa kola run basic` is equivalent to running: + +``` +kola -p qemu run basic --output-dir tmp/kola +``` + +## Automatic external test discovery + +When running inside a cosa workdir, kola automatically discovers external tests +from these locations (no `-E` flag required): + +1. **`src/config/tests/kola/`** -- tests from the config repo, registered with + the prefix `ext.config` (e.g., a test at + `src/config/tests/kola/files/aleph-version/test.sh` becomes + `ext.config.files.aleph-version`). +2. **`/usr/lib/coreos-assembler/tests/kola/`** -- tests installed by packages, + registered with the prefix `ext`. +3. **`./tests/kola/`** (current directory) -- if no `-E` flag is given and no + cosa build is detected, kola checks the current directory for a + `tests/kola/` subdirectory. + +The `-E`/`--exttest` flag is only needed for *additional* external test +directories beyond these defaults. See +[kola/external-tests.md](kola/external-tests.md) for details on how to write +external tests. + ## kola run The run command invokes the main kola test harness. It @@ -87,6 +127,22 @@ Example format of the file: The special pattern `skip-console-warnings` suppresses the default check for kernel errors on the console which would otherwise fail a test. +### Common flags + +| Flag | Description | +|------|-------------| +| `-j` / `--parallel N` | Run N tests in parallel (default: 1). Use `auto` to match CPU count. | +| `--tag TAG` | Run only tests with the given tag. May be repeated. Prefix with `!` to exclude. | +| `--denylist-test PATTERN` | Skip tests matching the pattern. May be repeated. | +| `-E` / `--exttest DIR` | Add an external test directory (see above for automatic discovery). | +| `--rerun` | Re-run failed tests once. | +| `--append-butane PATH` | Merge a Butane config snippet with every test's Ignition config. | +| `--append-ignition PATH` | Merge an Ignition config snippet with every test's Ignition config. | +| `--ssh-on-test-failure` | SSH into the machine when a test fails (useful for debugging). | +| `--multiply N` | Run matched tests N times (useful for finding race conditions). | + +Use `cosa kola run -h` for the full list of options. + ## kola list The list command lists all of the available tests. From 1b4ac39fb9a3d111526ff436cdce10976856071c Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 02:19:14 +0000 Subject: [PATCH 05/16] Add cosa-kola AI agent skill for running kola tests Add a skill that guides through the cosa kola test workflow: finding tests, running them, and checking results. The skill references docs/kola.md and docs/kola/external-tests.md for detailed flag and format documentation rather than duplicating them. Assisted-by: --- .agents/skills/cosa-kola/SKILL.md | 99 +++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 .agents/skills/cosa-kola/SKILL.md diff --git a/.agents/skills/cosa-kola/SKILL.md b/.agents/skills/cosa-kola/SKILL.md new file mode 100644 index 0000000000..d42a4502dd --- /dev/null +++ b/.agents/skills/cosa-kola/SKILL.md @@ -0,0 +1,99 @@ +--- +name: cosa-kola +description: Run kola tests against CoreOS builds using coreos-assembler, including native and external tests +--- + +# Running Tests with cosa kola + +This skill guides you through running kola tests against CoreOS builds using +coreos-assembler. + +## Out of scope + +Writing new native (Go) kola tests and cloud platform testing are not covered +here. For writing external tests, refer to the docs listed below. + +## Key documentation references + +Do NOT duplicate these docs. Read them at runtime for command details: + +| File | What it covers | +|------|----------------| +| `docs/kola.md` | Kola overview, `cosa kola` wrapper behavior, automatic test discovery, common flags, denylists, test output logs | +| `docs/kola/external-tests.md` | How to write and structure external tests, `kola.json` metadata, reboot support | +| `docs/kola/adding-tests.md` | Adding native Go tests to kola (for reference) | +| `src/cmd-kola` | `cosa kola` wrapper source -- shows how args are passed through to kola | + +## Prerequisites + +Before running tests you need a QEMU image in the cosa workdir. Either: + +- Build one: `cosa build && cosa osbuild qemu` +- Fetch one: `cosa buildfetch --stream stable --artifact qemu --decompress` + +See the cosa-basics skill for details on obtaining a build. + +## Workflow + +All commands below assume you are in a cosa workdir (or inside the cosa +container). Read `docs/kola.md` for full flag documentation. + +### Step 1: Find the right test(s) + +List available tests to find the one the user wants: + +``` +cosa kola list +``` + +For JSON output with descriptions: + +``` +cosa kola list --json | jq -r '.[] | [.Name, .Description] | @tsv' +``` + +External tests from `src/config/tests/kola/` are discovered automatically +and named `ext.config.` (e.g., `ext.config.files.aleph-version`). + +### Step 2: Run the test(s) + +Test names are glob patterns: + +``` +cosa kola run basic +cosa kola run ext.config.files.aleph-version +cosa kola run 'ext.config.*' +``` + +Running `cosa kola run` with no pattern runs all tests (can take a long time). + +Read `docs/kola.md` for flags like `--parallel`, `--tag`, `--rerun`, +`--denylist-test`, `--ssh-on-test-failure`, `--append-butane`, and others. + +### Step 3: Check results + +Test logs are written to `tmp/kola//`. Key files: `journal.txt`, +`console.txt`, `ignition.json`, `journal-raw.txt.gz`. Read `docs/kola.md` +("More information on tests") for details. + +## Special test types + +### ISO install tests + +ISO install tests are regular kola tests (the `testiso` subcommand was removed +in https://github.com/coreos/coreos-assembler/pull/4377). Run them with +`cosa kola run` like any other test: + +``` +cosa kola run iso.iso-offline-install.mpath.bios +cosa kola run 'iso.*' +``` + +### Upgrade tests + +``` +cosa kola --upgrades +``` + +This runs the `run-upgrade` subcommand, which tests OS upgrades. It +automatically finds parent images and caches them in `tmp/kola-qemu-cache/`. From 4a8bda8e8472433019be4dab8f3841678caa6a86 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 12:58:32 +0000 Subject: [PATCH 06/16] skills: cross-reference cosa-basics and cosa-kola cosa-kola's Prerequisites section duplicated build/fetch commands already covered by cosa-basics. Replace with a directive to load the cosa-basics skill, and add a reverse reference from cosa-basics to cosa-kola for running tests. Assisted-by: --- .agents/skills/cosa-basics/SKILL.md | 2 ++ .agents/skills/cosa-kola/SKILL.md | 10 ++++------ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.agents/skills/cosa-basics/SKILL.md b/.agents/skills/cosa-basics/SKILL.md index bc95a2333f..aa2d1808e7 100644 --- a/.agents/skills/cosa-basics/SKILL.md +++ b/.agents/skills/cosa-basics/SKILL.md @@ -169,3 +169,5 @@ auto-login as the `core` user. - The build directory at `$PWD` is mounted at `/srv/` inside the container. Build artifacts persist across container invocations. - The workdir's `builds/latest` symlink always points to the most recent build. +- To run kola tests against a build, load the **cosa-kola** skill. It covers + test discovery, `cosa kola run`, upgrade tests, and result inspection. diff --git a/.agents/skills/cosa-kola/SKILL.md b/.agents/skills/cosa-kola/SKILL.md index d42a4502dd..04fe17c60a 100644 --- a/.agents/skills/cosa-kola/SKILL.md +++ b/.agents/skills/cosa-kola/SKILL.md @@ -26,12 +26,10 @@ Do NOT duplicate these docs. Read them at runtime for command details: ## Prerequisites -Before running tests you need a QEMU image in the cosa workdir. Either: - -- Build one: `cosa build && cosa osbuild qemu` -- Fetch one: `cosa buildfetch --stream stable --artifact qemu --decompress` - -See the cosa-basics skill for details on obtaining a build. +Before running tests you need a QEMU image in the cosa workdir and the +`cosa()` bash alias must be set up. Load the **cosa-basics** skill for both +of these -- it covers container invocation, building from source, and fetching +published builds. ## Workflow From 0a592a4fdb7bf7d3e6ece90a480b9c27ff21c4d0 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 13:00:36 +0000 Subject: [PATCH 07/16] skills/cosa-kola: document how to find ext.config test sources Explain that ext.config.* test names map to paths under src/config/tests/kola/ in the cosa workdir. Include the dot-to-slash naming convention, which files to look for (test.sh, kola.json, config.bu), and concrete examples. Assisted-by: --- .agents/skills/cosa-kola/SKILL.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/.agents/skills/cosa-kola/SKILL.md b/.agents/skills/cosa-kola/SKILL.md index 04fe17c60a..07fc6f79aa 100644 --- a/.agents/skills/cosa-kola/SKILL.md +++ b/.agents/skills/cosa-kola/SKILL.md @@ -53,6 +53,28 @@ cosa kola list --json | jq -r '.[] | [.Name, .Description] | @tsv' External tests from `src/config/tests/kola/` are discovered automatically and named `ext.config.` (e.g., `ext.config.files.aleph-version`). +When you need to inspect or understand a test whose name starts with +`ext.config.`, find its source under `src/config/tests/kola/` in the cosa +workdir. The path maps directly from the test name: replace dots with `/` +after `ext.config.` and look for `test.sh` (and optionally `kola.json` or +`config.bu`) in that directory. For example: + +| Test name | Source path | +|-----------|-------------| +| `ext.config.files.aleph-version` | `src/config/tests/kola/files/aleph-version/test.sh` | +| `ext.config.upgrade.extended` | `src/config/tests/kola/upgrade/extended/test.sh` | + +The `src/config/` directory is the cloned config repo (e.g., +`fedora-coreos-config`). If it does not exist in the current directory, +check for it relative to the cosa workdir root. + +**Important:** The cosa workdir (created by `cosa init`) is NOT the same as +the coreos-assembler source repository. The `src/config/` directory only +exists inside a cosa workdir. Do not search the coreos-assembler source tree +for `ext.config.*` test files -- they won't be there. If you need to inspect +these test sources and are not currently in a cosa workdir, either navigate to +one or clone the config repo (e.g., `fedora-coreos-config`) directly. + ### Step 2: Run the test(s) Test names are glob patterns: From 41f781b00cee7e997299b7eaefd7e65b8955565f Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 13:22:26 +0000 Subject: [PATCH 08/16] skills/cosa-basics: --stream is optional with --build in buildfetch The stream is auto-detected from the build ID version number when --build is provided. Update the example and explanation to reflect this, including the version-to-stream mapping examples. Assisted-by: --- .agents/skills/cosa-basics/SKILL.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/.agents/skills/cosa-basics/SKILL.md b/.agents/skills/cosa-basics/SKILL.md index aa2d1808e7..ca38fcc7cb 100644 --- a/.agents/skills/cosa-basics/SKILL.md +++ b/.agents/skills/cosa-basics/SKILL.md @@ -122,12 +122,14 @@ If the user does not specify a stream, default to `stable`. **Fetch a specific build by ID:** ``` -cosa buildfetch --stream -b --artifact qemu --decompress +cosa buildfetch -b --artifact qemu --decompress ``` -The stream can be auto-detected from the build ID's version number (the third -dotted component maps to a stream), but providing `--stream` explicitly is -safer. Read `src/cmd-buildfetch` for the version-to-stream mapping. +When `-b`/`--build` is given, `--stream` is not required -- the stream is +auto-detected from the build ID's version number (the third dotted component +maps to a stream, e.g. `.2.` = `testing`, `.3.` = `stable`). Read +`src/cmd-buildfetch` for the full version-to-stream mapping. If `--stream` +is also provided it must match or the command will error. **If the user wants a non-qemu artifact:** replace `--artifact qemu` with the appropriate artifact name (e.g., `metal`, `aws`, `gcp`). But note that cloud From cc505f9b77b05db39df867437e151a1abdcb4459 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 14:56:33 +0000 Subject: [PATCH 09/16] docs: replace buildextend references with osbuild/imageupload The buildextend-* commands are deprecated symlinks. Disk images are now built with 'cosa osbuild ' and uploaded to cloud providers with 'cosa imageupload-'. - Rename docs/cosa/buildextend-aws.md to imageupload-aws.md - Rename docs/cosa/buildextend-secex.md to osbuild-secex.md - Update docs/design.md, docs/kola.md, docs/working.md, and mantle/README.md to use the new command names - Add imageupload-* commands to docs/cosa.md command reference - Rename 'Extended artifacts' section in docs/kola.md to 'Testing platform artifacts' Assisted-by: --- docs/cosa.md | 13 +++++++++++++ .../cosa/{buildextend-aws.md => imageupload-aws.md} | 9 ++++++--- .../cosa/{buildextend-secex.md => osbuild-secex.md} | 6 +++--- docs/design.md | 7 ++++--- docs/kola.md | 8 ++++---- docs/working.md | 2 +- mantle/README.md | 2 +- 7 files changed, 32 insertions(+), 15 deletions(-) rename docs/cosa/{buildextend-aws.md => imageupload-aws.md} (90%) rename docs/cosa/{buildextend-secex.md => osbuild-secex.md} (90%) diff --git a/docs/cosa.md b/docs/cosa.md index 27b8e9ead4..008df0e4e6 100644 --- a/docs/cosa.md +++ b/docs/cosa.md @@ -38,6 +38,19 @@ various platforms, such as cloud providers. | `osbuild --supported-platforms` | List of all supported platforms +## Image upload commands + +Upload disk images to cloud providers. Build the platform image first with +`cosa osbuild `. + +| Name | Description | +| ---- | ----------- | +| `imageupload-aliyun` | Upload an image to Alibaba Cloud (Aliyun) | +| `imageupload-aws` | Upload an image to AWS as an AMI | +| `imageupload-azure` | Upload an image to Azure | +| `imageupload-gcp` | Upload an image to Google Cloud Platform | +| `imageupload-powervs` | Upload an image to IBM Power Virtual Server | + ## Misc commands Those less commonly used commands are listed here: diff --git a/docs/cosa/buildextend-aws.md b/docs/cosa/imageupload-aws.md similarity index 90% rename from docs/cosa/buildextend-aws.md rename to docs/cosa/imageupload-aws.md index f9ac3d94b4..5364aad59a 100644 --- a/docs/cosa/buildextend-aws.md +++ b/docs/cosa/imageupload-aws.md @@ -3,9 +3,10 @@ parent: CoreOS Assembler Command Line Reference nav_order: 1 --- -# cosa buildextend-aws +# cosa imageupload-aws -Using `cosa buildextend-aws` looks for AWS credentials in the standard places; +`cosa imageupload-aws` uploads a disk image to AWS as an AMI. It looks for +AWS credentials in the standard places; Common methods are `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables, and using `AWS_PROFILE`. For more information, consult the documentation for the `aws` CLI tool, in particular @@ -86,5 +87,7 @@ The full list of permisions required for your IAM policy should look similar to } ``` -This is currently a wrapper around the `ore` subcommand of mantle; there is an +Before uploading, build the AWS disk image with `cosa osbuild aws`. + +This command is a wrapper around the `ore` subcommand of mantle; there is an `ore aws initialize` subcommand which may be useful. diff --git a/docs/cosa/buildextend-secex.md b/docs/cosa/osbuild-secex.md similarity index 90% rename from docs/cosa/buildextend-secex.md rename to docs/cosa/osbuild-secex.md index 5eaa6a63a0..70025f4eb8 100644 --- a/docs/cosa/buildextend-secex.md +++ b/docs/cosa/osbuild-secex.md @@ -3,9 +3,9 @@ parent: CoreOS Assembler Command Line Reference nav_order: 1 --- -# cosa buildextend-secex +# cosa osbuild qemu-secex -This buildextend command is used to build QEMU images that are enabled for IBM Secure Execution on IBM Z. +This command builds QEMU images that are enabled for IBM Secure Execution on IBM Z. In order to build a QEMU image protected by IBM Secure Execution, you need to provide a host key to encrypt it. For more information on IBM Secure Execution on IBM Z, refer to the [IBM Documentation](https://www.ibm.com/docs/en/linux-on-systems?topic=ibmz-secure-execution). @@ -18,7 +18,7 @@ This results in a few specifics to note: - The firstboot service will fail when no host key is provided, as the sdboot-image can not be recreated. - Write the host key(s) to: `/etc/se-hostkeys/ibm-z-hostkey-.crt` -To facilitate this, `buildextend-secex` can take 2 mutually exclusive additional arguments: `--genprotimgvm ` and `--hostkey `. +To facilitate this, `cosa osbuild qemu-secex` can take 2 mutually exclusive additional arguments: `--genprotimgvm ` and `--hostkey `. If none is provided, `--genprotimgvm` is used with default values. ## `--genprotimgvm ` (default) diff --git a/docs/design.md b/docs/design.md index 6b5864b771..0afee62d5a 100644 --- a/docs/design.md +++ b/docs/design.md @@ -37,9 +37,10 @@ commit. There is also a `builds/builds.json` which maintains the list of builds. The reason for this is that HTTP doesn't offer a way to enumerate a directory. -After a build is generated there are a variety of `buildextend-$x` commands, -for example `buildextend-ec2` which can upload to AWS, and `buildextend-metal` -which generates a bare metal disk image. +After a build is generated, `cosa osbuild ` creates disk images for +specific platforms (e.g. `cosa osbuild metal` for bare metal, `cosa osbuild aws` +for AWS). To upload images to cloud providers, use `cosa imageupload-` +(e.g. `cosa imageupload-aws`). By default, builds are pruned (as is the OSTree repository), although one can use `build --no-prune` to prevent this. diff --git a/docs/kola.md b/docs/kola.md index 69d3bd7656..594abb66a5 100644 --- a/docs/kola.md +++ b/docs/kola.md @@ -254,12 +254,12 @@ After you run the kola test, you can find more information in `tmp/kola/` Is the most generic way of testing extended artifacts, this is mostly useful for the cloud platforms +3. `cosa osbuild ` (Build a disk image for a specific platform, e.g. `cosa osbuild aws`) +4. `kola run -p ` Is the most generic way of testing platform artifacts, this is mostly useful for the cloud platforms 5. For running the likes of metal/metal4k artifacts there's not much difference than running `kola run` from the coreos-assembler 6. `cd builds/latest/` (This will show your latest build information) 7. `cosa list` (This will show you the most recent CoreOS builds that have been made and the artifacts that were created) diff --git a/docs/working.md b/docs/working.md index a5fd78aadb..ada5e15850 100644 --- a/docs/working.md +++ b/docs/working.md @@ -357,7 +357,7 @@ requires access to podman and /dev/kvm. # to run the freshly built image cosa run # to build the live ISO - cosa buildextend-metal && cosa buildextend-live + cosa osbuild metal live # to run the live ISO cosa run -p qemu-iso ``` diff --git a/mantle/README.md b/mantle/README.md index ff5ad4a3aa..36724179c0 100644 --- a/mantle/README.md +++ b/mantle/README.md @@ -159,7 +159,7 @@ api for each cloud provider closely, so the interface for each cloud provider is different. See each providers `help` command for the available actions. Note, when uploading to cloud platforms, be sure to use the image built for -that particular platform (with `cosa buildextend-...`). +that particular platform (with `cosa osbuild `). ### plume Plume is the Container Linux release utility. Releases are done in two stages, From 33583c0b77a0357cb1acf492a986d365677a8198 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 15:12:44 +0000 Subject: [PATCH 10/16] docs/cosa/run: document kola spawn for cloud platforms cosa run is QEMU-only. Add a section pointing users to 'cosa kola spawn -p ' for spawning instances on cloud platforms, with examples for AWS and GCP. Assisted-by: --- docs/cosa/run.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/cosa/run.md b/docs/cosa/run.md index af9557b353..2bce573c9a 100644 --- a/docs/cosa/run.md +++ b/docs/cosa/run.md @@ -17,6 +17,21 @@ $ cosa run --qemu-image rhcos-qemu.qcow2 Many additional options are supported. Use `--help` to see them. The following sections discuss a few in more details. +## Running on cloud platforms + +`cosa run` only supports local QEMU VMs. To spawn instances on cloud +platforms, use `kola spawn` with the `-p` flag: + +``` +$ cosa kola spawn -p aws --aws-ami --aws-region us-east-1 +$ cosa kola spawn -p gcp --gcp-image --gcp-json-key +``` + +This launches an instance and opens an SSH shell. Use `--remove=false` to +keep the instance running after you exit, or `--detach` to spawn in the +background. See `cosa kola spawn --help` for all options and +[credentials](../mantle/credentials.md) for platform authentication setup. + ## Testing Butane or Ignition configs Using `cosa run` is a very effective way to iterate on your Butane or Ignition From 9909c4e2a57f7d2a7f73da636d26aa7862cd43de Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 15:13:42 +0000 Subject: [PATCH 11/16] Add cosa-platforms AI agent skill New skill covering non-QEMU platform workflows: building disk images with 'cosa osbuild', uploading via 'cosa imageupload-*', running kola tests with '-p ', and spawning cloud instances with 'kola spawn'. Includes a platform reference table, credential pointers, and workflow steps. Assisted-by: --- .agents/skills/cosa-platforms/SKILL.md | 150 +++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 .agents/skills/cosa-platforms/SKILL.md diff --git a/.agents/skills/cosa-platforms/SKILL.md b/.agents/skills/cosa-platforms/SKILL.md new file mode 100644 index 0000000000..5069b836b0 --- /dev/null +++ b/.agents/skills/cosa-platforms/SKILL.md @@ -0,0 +1,150 @@ +--- +name: cosa-platforms +description: Build disk images, upload to cloud providers, and run tests or spawn instances on non-QEMU platforms (AWS, GCP, Azure, ISO, metal, etc.) +--- + +# Platform Images and Cloud Testing with cosa + +This skill covers working with non-QEMU platforms in coreos-assembler: +building platform-specific disk images, uploading them to cloud providers, +running kola tests on cloud platforms, and spawning interactive cloud +instances. + +## Related skills + +- Load the **cosa-basics** skill for workdir setup, the `cosa()` bash alias, + and general build workflow. +- Load the **cosa-kola** skill for general kola test mechanics (test + discovery, result inspection, common flags, external tests). +- This skill adds the platform-specific details on top of those. + +## Key documentation references + +Do NOT duplicate these docs. Read them at runtime for command details: + +| File | What it covers | +|------|----------------| +| `docs/mantle/credentials.md` | Credential setup for all platforms (AWS, Azure, GCP, OpenStack, DO, ESX, etc.) | +| `docs/cosa.md` | Command reference including `osbuild` and `imageupload-*` | +| `docs/cosa/imageupload-aws.md` | AWS AMI upload (IAM permissions, credentials) | +| `docs/cosa/osbuild-secex.md` | IBM Secure Execution image building | +| `docs/cosa/run.md` | `cosa run` (QEMU-only) and `kola spawn` for cloud instances | +| `docs/kola.md` ("Run tests on cloud platforms") | Cloud testing examples for AWS, GCP, Azure | +| `mantle/cmd/kola/options.go` | Source of truth for all platform-specific flags | + +## Supported platforms + +Kola supports the following platforms via the `-p`/`--platform` flag: + +| Platform | `-p` value | Image flag | Default credentials | +|----------|-----------|------------|-------------------| +| AWS | `aws` | `--aws-ami` | `~/.aws/credentials` | +| Azure | `azure` | `--azure-disk-uri` | `~/.azure/azureCreds.json` | +| DigitalOcean | `do` | `--do-image` | `~/.config/digitalocean.json` | +| ESX/VMware | `esx` | `--esx-base-vm` | `~/.config/esx.json` | +| GCP | `gcp` | `--gcp-image` | GCP service account JSON key | +| OpenStack | `openstack` | `--openstack-image` | `~/.config/openstack.json` | +| QEMU | `qemu` | `--qemu-image` | (none) | +| QEMU ISO | `qemu-iso` | `--qemu-iso` | (none) | + +Read `docs/mantle/credentials.md` for full credential setup details per +platform, and `mantle/cmd/kola/options.go` or `cosa kola run -h` for the +complete flag list. + +## Workflow + +### Step 1: Build the platform image + +``` +cosa osbuild +``` + +Use `cosa osbuild --supported-platforms` to list all available platforms. +Multiple platforms can be built in one invocation: + +``` +cosa osbuild aws azure gcp +``` + +This creates disk images in the appropriate format for each platform (e.g. +VMDK for AWS, VHD for Azure, tar.gz for GCP, raw for metal). + +### Step 2: Upload to the cloud provider + +For cloud platforms that require image upload: + +``` +cosa imageupload- +``` + +Available upload targets: `aliyun`, `aws`, `azure`, `gcp`, `powervs`. + +Read `docs/cosa/imageupload-aws.md` for AWS-specific IAM and credential +setup. Not all platforms have an `imageupload-*` command; for those that +don't, use `ore` directly or upload manually. + +### Step 3: Set up credentials + +Read `docs/mantle/credentials.md` for per-platform credential configuration. +Each platform has its own credential file format and default location (see +the table above). + +### Step 4: Run tests on a cloud platform + +``` +cosa kola run -p [platform-specific flags] +``` + +Examples: + +``` +# AWS +cosa kola run -p aws --aws-ami --aws-region us-east-1 basic + +# GCP +cosa kola run -p gcp --gcp-image --gcp-json-key --gcp-project basic + +# Azure +cosa kola run -p azure --azure-credentials --azure-disk-uri basic +``` + +Read `docs/kola.md` ("Run tests on cloud platforms") for more examples. +For the full list of flags per platform, read `mantle/cmd/kola/options.go` +or run `cosa kola run -h`. + +### Step 5: Spawn instances for interactive debugging + +``` +cosa kola spawn -p [platform-specific flags] +``` + +This launches an instance and opens an SSH shell without running tests. +Useful for interactive debugging. Key flags: + +| Flag | Purpose | +|------|---------| +| `--shell` / `-s` | Open an SSH shell (default: true) | +| `--detach` / `-t` | Spawn in the background (implies `--shell=false --remove=false --verbose --keys`) | +| `--remove=false` | Keep the instance running after the shell exits | +| `--keys` / `-k` | Inject SSH keys from the agent and `~/.ssh/` | +| `--nodecount` / `-c` | Number of instances to spawn | +| `--userdata` / `-u` | Path to Ignition/Butane config for the instance | +| `--ssh-command` / `-x` | Run a command over SSH instead of opening a shell | + +Note: `--reconnect` is only available on QEMU platforms. + +Read `docs/cosa/run.md` ("Running on cloud platforms") for more details. + +## Important notes + +- `cosa run` is QEMU-only. For all other platforms, use + `cosa kola run -p ` (for tests) or + `cosa kola spawn -p ` (for interactive instances). +- When a cosa build exists in the workdir, kola auto-fills some platform + image references from the build metadata (e.g. AMI for AWS, image name + for GCP). This means you may not need to specify `--aws-ami` etc. if the + build already has that info in `meta.json`. +- Instance types default to reasonable values per platform and architecture + (e.g. `m5.large` for AWS x86_64, `c6g.xlarge` for aarch64). Override with + platform-specific type flags (e.g. `--aws-type`, `--gcp-machinetype`, + `--azure-size`). From 7cf494df5d5ffa6c3c39c10a3a78611c631d192f Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 15:14:18 +0000 Subject: [PATCH 12/16] skills: reference cosa-platforms from cosa-basics and cosa-kola Update cosa-basics to point to the actual cosa-platforms skill name instead of the placeholder 'cosa-cloud'. Add a directive in cosa-kola to load cosa-platforms when testing on non-QEMU platforms. Assisted-by: --- .agents/skills/cosa-basics/SKILL.md | 2 +- .agents/skills/cosa-kola/SKILL.md | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/.agents/skills/cosa-basics/SKILL.md b/.agents/skills/cosa-basics/SKILL.md index ca38fcc7cb..83d0f889b5 100644 --- a/.agents/skills/cosa-basics/SKILL.md +++ b/.agents/skills/cosa-basics/SKILL.md @@ -12,7 +12,7 @@ or by fetching a published build), and launching a local QEMU VM. ## Out of scope Cloud platform image building and uploading (AWS, GCP, Azure, etc.) are not -covered here. Use a dedicated cosa-cloud skill for those workflows. +covered here. Load the **cosa-platforms** skill for those workflows. ## Key documentation references diff --git a/.agents/skills/cosa-kola/SKILL.md b/.agents/skills/cosa-kola/SKILL.md index 07fc6f79aa..23e96c1dd3 100644 --- a/.agents/skills/cosa-kola/SKILL.md +++ b/.agents/skills/cosa-kola/SKILL.md @@ -90,6 +90,9 @@ Running `cosa kola run` with no pattern runs all tests (can take a long time). Read `docs/kola.md` for flags like `--parallel`, `--tag`, `--rerun`, `--denylist-test`, `--ssh-on-test-failure`, `--append-butane`, and others. +For running tests on platforms other than QEMU (e.g. AWS, GCP, Azure, ISO, +metal), load the **cosa-platforms** skill. + ### Step 3: Check results Test logs are written to `tmp/kola//`. Key files: `journal.txt`, From 116e1dd19205fa2e6cf556a165817d6103c6484e Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 15:20:23 +0000 Subject: [PATCH 13/16] skills: document lightweight buildfetch for cloud platform testing When cloud images already exist for a build (e.g. AMIs uploaded to AWS), there is no need to fetch disk artifacts or decompress them. A plain 'cosa buildfetch -b ' fetches only meta.json, which contains the cloud image references that kola picks up automatically. Update cosa-platforms with a dedicated note explaining this shortcut and update cosa-basics to clarify when --artifact/--decompress are needed vs. when they can be skipped. Assisted-by: --- .agents/skills/cosa-basics/SKILL.md | 16 +++++++++++----- .agents/skills/cosa-platforms/SKILL.md | 21 +++++++++++++++++---- 2 files changed, 28 insertions(+), 9 deletions(-) diff --git a/.agents/skills/cosa-basics/SKILL.md b/.agents/skills/cosa-basics/SKILL.md index 83d0f889b5..4e97173566 100644 --- a/.agents/skills/cosa-basics/SKILL.md +++ b/.agents/skills/cosa-basics/SKILL.md @@ -104,10 +104,16 @@ This downloads a previously published build from the Fedora CoreOS build servers. Use this when you want to test or inspect an existing release without building from source. -**Default flags:** Always include `--artifact qemu --decompress` unless the -user specifically asks for a different platform artifact. The published -artifacts are stored compressed on the server; `--decompress` expands them so -`cosa run` can use the resulting qcow2 directly. +**Default flags:** Include `--artifact qemu --decompress` when the user +needs a local QEMU disk image (e.g. for `cosa run` or QEMU-based kola +tests). The published artifacts are stored compressed on the server; +`--decompress` expands them so the resulting qcow2 can be used directly. + +If the user only needs to run tests on a cloud platform where images already +exist (e.g. AMIs in AWS), `--artifact` and `--decompress` are not needed. +A plain `cosa buildfetch -b ` fetches only a few small metadata +files (including `meta.json`), which contain cloud image references that +kola picks up automatically. See the **cosa-platforms** skill for details. **Fetch the latest build for a stream:** @@ -133,7 +139,7 @@ is also provided it must match or the command will error. **If the user wants a non-qemu artifact:** replace `--artifact qemu` with the appropriate artifact name (e.g., `metal`, `aws`, `gcp`). But note that cloud -platform workflows are better served by a dedicated cosa-cloud skill. +platform workflows are better served by the **cosa-platforms** skill. ### Step 3: Launch the VM diff --git a/.agents/skills/cosa-platforms/SKILL.md b/.agents/skills/cosa-platforms/SKILL.md index 5069b836b0..121352e252 100644 --- a/.agents/skills/cosa-platforms/SKILL.md +++ b/.agents/skills/cosa-platforms/SKILL.md @@ -140,10 +140,23 @@ Read `docs/cosa/run.md` ("Running on cloud platforms") for more details. - `cosa run` is QEMU-only. For all other platforms, use `cosa kola run -p ` (for tests) or `cosa kola spawn -p ` (for interactive instances). -- When a cosa build exists in the workdir, kola auto-fills some platform - image references from the build metadata (e.g. AMI for AWS, image name - for GCP). This means you may not need to specify `--aws-ami` etc. if the - build already has that info in `meta.json`. +- When a cosa build exists in the workdir, kola auto-fills platform image + references from the build metadata (e.g. AMI for AWS, image name for GCP). + This means you may not need to specify `--aws-ami` etc. if the build + already has that info in `meta.json`. Check `builds///meta.json` + to see if the cloud image info is present. +- If images already exist in the cloud for a build, you can skip both + `cosa osbuild` and `cosa imageupload-*` entirely. Just fetch the build + metadata with a lightweight `cosa buildfetch` (no `--artifact` or + `--decompress` needed): + ``` + cosa buildfetch -b + ``` + Without `--artifact`, this only downloads a few small metadata files + (`meta.json`, `commitmeta.json`, etc.) -- not any disk images. The + `meta.json` contains the cloud image references (AMI IDs, GCP image + names, etc.) and kola will pick them up automatically when you run + `cosa kola run -p `. - Instance types default to reasonable values per platform and architecture (e.g. `m5.large` for AWS x86_64, `c6g.xlarge` for aarch64). Override with platform-specific type flags (e.g. `--aws-type`, `--gcp-machinetype`, From ce046dc211b35a6825ba951c66bd952630b3dee6 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Fri, 29 May 2026 17:45:01 +0000 Subject: [PATCH 14/16] Add cosa-dev skill for developing coreos-assembler itself Covers three workflows for iterating on cosa code: 1. Building mantle Go binaries (kola, ore, plume) inside the cosa container and mounting them over the installed copies for testing, with persistent Go build cache for fast incremental rebuilds. 2. Using COREOS_ASSEMBLER_GIT to mount modified Python/shell scripts from src/ without any rebuild. 3. Full container rebuild via podman build for Dockerfile or dependency changes. Assisted-by: --- .agents/skills/cosa-dev/SKILL.md | 233 +++++++++++++++++++++++++++++++ 1 file changed, 233 insertions(+) create mode 100644 .agents/skills/cosa-dev/SKILL.md diff --git a/.agents/skills/cosa-dev/SKILL.md b/.agents/skills/cosa-dev/SKILL.md new file mode 100644 index 0000000000..6eb0f61e41 --- /dev/null +++ b/.agents/skills/cosa-dev/SKILL.md @@ -0,0 +1,233 @@ +--- +name: cosa-dev +description: Develop and test changes to coreos-assembler itself -- build mantle Go binaries (kola, ore, plume), iterate on Python/shell scripts, or do full container rebuilds +--- + +# Developing coreos-assembler + +This skill covers workflows for making and testing changes to +coreos-assembler (COSA) itself, including the Go binaries in `mantle/`, +the Python/shell scripts in `src/`, and the container image. + +## Related skills + +- Load the **cosa-basics** skill for the `cosa()` bash alias, workdir + setup, and general build workflow. +- Load the **cosa-kola** skill for running kola tests. +- Load the **cosa-platforms** skill for cloud platform testing. + +## Key documentation references + +Do NOT duplicate these docs. Read them at runtime for command details: + +| File | What it covers | +|------|----------------| +| `docs/devel.md` | Developer workflows: hacking scripts, building Go binaries, container rebuilds | +| `docs/building-fcos.md` | The `cosa()` bash alias definition | +| `Makefile` | Build targets: `mantle`, `kola`, `ore`, `plume`, `kolet`, `install` | +| `mantle/build` | Mantle build script (called by `make kola` etc.) | +| `build.sh` | Container image build steps | +| `Dockerfile` | Container image definition | + +## Choosing an approach + +| What changed | Approach | Rebuild time | +|---|---|---| +| Go code in `mantle/` only | Build single binary, mount over container copy | ~2 min cold, seconds warm | +| Python/shell in `src/` only | Set `COREOS_ASSEMBLER_GIT` | Instant (no rebuild) | +| Both Go and Python/shell | Combine both approaches | ~2 min cold, seconds warm | +| Dockerfile, RPM deps, or everything | Full `podman build` | Several minutes | + +## Approach 1: Modifying mantle Go binaries (kola, ore, plume) + +Use this when changing Go code under `mantle/`. The idea is to build the +binary inside the cosa container (which has the Go toolchain), write it +to a host directory, then mount it over the container's installed copy on +subsequent runs. + +### Buildable binaries + +| Source path | Binary name | Installed at | +|---|---|---| +| `mantle/cmd/kola` | `kola` | `/usr/bin/kola` | +| `mantle/cmd/ore` | `ore` | `/usr/bin/ore` | +| `mantle/cmd/plume` | `plume` | `/usr/bin/plume` | +| `mantle/cmd/kolet` | `kolet` | `/usr/lib/kola//kolet` | + +### Step 1: Create output and cache directories + +``` +mkdir -p /tmp/cosa-go-cache /tmp/cosa-bin +chmod 777 /tmp/cosa-go-cache /tmp/cosa-bin +``` + +The `chmod 777` is needed because the cosa container runs as uid 1000 +(`builder`) via `--userns=keep-id`, and the directories must be writable +by that mapped user. + +### Step 2: Build the binary + +Mount the cosa source tree, the output directory, and optionally a +persistent Go build cache into the container: + +``` +export COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS="\ + -v=/path/to/coreos-assembler:/srv/cosa-src:ro \ + -v=/tmp/cosa-bin:/srv/cosa-bin:rw \ + -v=/tmp/cosa-go-cache:/home/builder/.cache/go-build:rw" + +cosa shell -- bash -c \ + 'cd /srv/cosa-src && go build -buildvcs=false -o /srv/cosa-bin/kola ./mantle/cmd/kola' +``` + +Replace `kola` with `ore` or `plume` as needed. To build all mantle +binaries at once: + +``` +cosa shell -- bash -c \ + 'cd /srv/cosa-src && \ + for cmd in kola ore plume; do \ + go build -buildvcs=false -o /srv/cosa-bin/$cmd ./mantle/cmd/$cmd; \ + done' +``` + +**Key flags:** + +- `-buildvcs=false` -- required to avoid git ownership errors inside the + container (the source tree is mounted from the host with a different uid). +- The persistent Go build cache (`/home/builder/.cache/go-build`) makes + subsequent rebuilds incremental. The first build downloads and compiles + all dependencies (~2 minutes); rebuilds after a small code change take + only a few seconds. + +### Step 3: Quick compilation check (no binary output) + +To verify that the code compiles without producing a binary: + +``` +cosa shell -- bash -c \ + 'cd /srv/cosa-src && go build -buildvcs=false ./mantle/...' +``` + +### Step 4: Test with the modified binary + +Mount the built binary over the container's installed copy using +`COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS`: + +``` +export COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS="\ + -v=/tmp/cosa-bin/kola:/usr/bin/kola:ro" + +cosa kola run -p qemu basic +``` + +This can be combined with other mounts. For example, to also mount +AWS credentials: + +``` +export COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS="\ + -v=/tmp/cosa-bin/kola:/usr/bin/kola:ro \ + -v=/path/to/aws-creds:/srv/aws-creds:ro" + +cosa kola run -p aws --aws-credentials-file /srv/aws-creds basic +``` + +### Step 5: Iterate + +After making further code changes, re-run the build command from Step 2. +With the persistent Go cache, only the changed packages are recompiled. +Then re-run the test from Step 4 -- the mount picks up the new binary +automatically. + +## Approach 2: Modifying Python/shell scripts (src/) + +Use this when changing files under `src/` (e.g. `src/cmd-build`, +`src/cmd-init`, `src/cosalib/*.py`). No rebuild is needed. + +Set the `COREOS_ASSEMBLER_GIT` environment variable to point at your +local coreos-assembler checkout. The `cosa()` bash alias will +automatically mount `$COREOS_ASSEMBLER_GIT/src/` over +`/usr/lib/coreos-assembler/` inside the container: + +``` +export COREOS_ASSEMBLER_GIT=/path/to/coreos-assembler +cosa init https://github.com/coreos/fedora-coreos-config +cosa build +``` + +Changes to files under `src/` take effect immediately on the next `cosa` +invocation. No rebuild or remount is required. + +**Limitation:** This only covers files installed to +`/usr/lib/coreos-assembler/` (i.e. `src/` contents). It does NOT affect +Go binaries (`kola`, `ore`, `plume`) or system packages. For Go changes, +combine this with Approach 1. + +### Combining with Go binary changes + +``` +export COREOS_ASSEMBLER_GIT=/path/to/coreos-assembler +export COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS="\ + -v=/tmp/cosa-bin/kola:/usr/bin/kola:ro" + +cosa kola run basic +``` + +This uses the modified Python/shell scripts from `COREOS_ASSEMBLER_GIT` +AND the modified kola binary from the mount. + +## Approach 3: Full container rebuild + +Use this when changing the `Dockerfile`, adding or updating RPM +dependencies, or when you want to verify everything works together as a +complete image. + +### Build the container image + +From the coreos-assembler repo root: + +``` +podman build -t localhost/coreos-assembler . +``` + +To speed things up by reusing the official image as a base (useful when +only cosa code changed, not dependencies): + +``` +podman build -t localhost/coreos-assembler . \ + --from quay.io/coreos-assembler/coreos-assembler:latest +``` + +### Use the locally-built container + +Set `COREOS_ASSEMBLER_CONTAINER` so the `cosa()` alias uses your local +image instead of the upstream one: + +``` +export COREOS_ASSEMBLER_CONTAINER=localhost/coreos-assembler +cosa init https://github.com/coreos/fedora-coreos-config +cosa build +``` + +This is the slowest approach but the most complete. It is needed when: + +- The `Dockerfile` or `build.sh` changed. +- System RPM dependencies were added or updated. +- You want a final integration check before submitting a PR. + +## Important notes + +- The `cosa()` bash alias creates a **transient container** for each + invocation. All state persists in the cosa workdir (mounted at + `/srv/`), not inside the container. This is why mounting files in via + `COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS` works -- each run gets the + latest version of whatever is mounted. +- Use `cosa shell` to get a persistent interactive session inside the + container. This is useful for running multiple build/test commands + without container startup overhead. However, be aware that mounted + binaries are fixed at container start time. +- The Go build cache directory inside the container is + `/home/builder/.cache/go-build`. Persisting it to the host avoids + cold rebuilds. +- When mounting binaries with `:ro`, the container cannot modify them. + This is intentional -- it prevents accidental overwrites. From 0d607c2a63b63f22ab73b63227ade883e2c6018f Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Mon, 1 Jun 2026 16:28:13 -0400 Subject: [PATCH 15/16] skills/cosa-basics: document Anubis use by Fedora Infrastructure This should help getting blocked when querying Fedora's infra for information. --- .agents/skills/cosa-basics/SKILL.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.agents/skills/cosa-basics/SKILL.md b/.agents/skills/cosa-basics/SKILL.md index 4e97173566..a18c79b207 100644 --- a/.agents/skills/cosa-basics/SKILL.md +++ b/.agents/skills/cosa-basics/SKILL.md @@ -179,3 +179,7 @@ auto-login as the `core` user. - The workdir's `builds/latest` symlink always points to the most recent build. - To run kola tests against a build, load the **cosa-kola** skill. It covers test discovery, `cosa kola run`, upgrade tests, and result inspection. +- Fedora infrastructure services (like Bodhi or Koji) are behind Anubis to + prevent abuse. When querying APIs or web pages for information set a + User-Agent and Accept headers to increase the chance of making it + through unchallenged. From 795420ffb663b939cdf05b3b0070ef6007bf21f2 Mon Sep 17 00:00:00 2001 From: Dusty Mabe Date: Mon, 1 Jun 2026 16:35:38 -0400 Subject: [PATCH 16/16] docs: document how to use AI skills --- docs/using-ai-skills.md | 43 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 docs/using-ai-skills.md diff --git a/docs/using-ai-skills.md b/docs/using-ai-skills.md new file mode 100644 index 0000000000..d302e82ddb --- /dev/null +++ b/docs/using-ai-skills.md @@ -0,0 +1,43 @@ +--- +nav_order: 10 +--- + +# Working with COSA's AI skills +{: .no_toc } + +Example tasks to hand to an AI tool (i.e. opencode, goose) leveraging +the skills under .agents/skills. To leverage a skill run `/skills` and +choose a skill and then give a prompt like the examples below: + +1. TOC +{:toc} + +## Simple Task + + Run the ext.config.upgrade.extended test starting from 44.20260510.2.1. When done look at the + journal in the logs from the test and tell me how long the rpm-ostree operations took for every boot. + +## Intermediate Task + + Do a local build against rawhide, but override the kernel in the build with the kernel build from + this bodhi update https://bodhi.fedoraproject.org/updates/FEDORA-2026-62bf55e4d8 and run the + ext.config.security.lockdown test against the built image. + +## Advanced Task + + With the local changes to the git repo here compile a new kola. Then write a new external test that + is a minimal version of ext.config.upgrade.extended but really only does: + + 1. leaves zincati disabled + 2. does a skopeo copy of quay.io/fedora/fedora-coreos:stable to an ociarchive + 3. does a manual `sudo rpm-ostree rebase ostree-unverified-image:oci-archive://path/to/ociarchive` + + Then we should run kola (newly compiled) to start an instance at the earliest `stable` stream build + based on Fedora 43 and run the test. + + I want to run the test with and without one change to benchmark them and compare the differences: + + - updating fsync to false in the ostree repo config (see https://ostreedev.github.io/ostree/man/ostree.repo-config.html) + inside the instance + + Run this test on AWS m8a.xlarge instances using the credentials in the `aws-creds` file.