From 6b7b319ca692c69e0bce94b59bbf358538e61a2f Mon Sep 17 00:00:00 2001 From: avivklas <> Date: Wed, 30 Sep 2026 14:14:31 +0300 Subject: [PATCH] docs: rewrite README to be more approachable and discoverable Co-authored-by: Cursor --- README.md | 382 ++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 243 insertions(+), 139 deletions(-) diff --git a/README.md b/README.md index 7cd1436..12e1a5f 100644 --- a/README.md +++ b/README.md @@ -2,117 +2,99 @@ fender logo

-# fender +

fender

-**fender** is a transparent Docker Unix socket proxy that frees you from the implicit Docker Hub registry lock-in — without touching Dockerfiles, CI scripts, or CLI habits. - -It works by sitting between the Docker CLI and the Docker daemon. On startup it registers itself as a Docker context and activates it, so all Docker tooling routes through fender automatically. When you shut fender down it removes its context and restores whatever you had before. +

+ Fix Docker Hub rate limits without touching your Dockerfiles.
+ Redirect every docker pull, docker run, and docker build to your own registry mirror, locally or in CI. +

-``` -docker pull nginx:latest # you type this - │ - ▼ Docker context: "fender" (~/.fender/fender.sock) - ┌─────────────────────────────────────────────────────────┐ - │ fender │ - │ docker.io/library/nginx:latest │ - │ ↓ rewrite │ - │ registry.example.com/library/nginx:latest │ - └─────────────────────────────────────────────────────────┘ - │ - ▼ upstream: active Docker context before fender started - Docker Daemon -``` +

+ Latest release + CI + Go version + MIT license +

--- -## Why fender? +Does this look familiar? -* **Bypass Docker Hub Rate Limits (`429 Too Many Requests`)**: Seamlessly route pulls through an internal mirror (Harbor, Sonatype Nexus, JFrog Artifactory, AWS ECR pull-through cache, GCP Artifact Registry, GitLab Dependency Proxy) in CI/CD without rewriting existing Dockerfiles or pipeline scripts. -* **No `daemon.json` or root access required**: Docker's native `registry-mirrors` requires modifying `/etc/docker/daemon.json` and restarting `dockerd`—impossible on GitHub-hosted runners or locked-down environments. `fender` runs purely in user space by registering an ephemeral Docker context. -* **Mirrors any registry, not just Docker Hub**: Unlike Docker's built-in mirror feature (which only supports `docker.io`), `fender` rewrites arbitrary registries like `ghcr.io` or `quay.io` via `registry_map`. -* **Zero changes to legacy codebases**: Developers keep typing `docker pull nginx` or writing `FROM python:3.11-slim`. No need to mass-edit hundreds of repositories across teams. -* **Transparent BuildKit support**: Intercepts `docker build` (`DOCKER_BUILDKIT=1`) gRPC `Solve` calls with an embedded frontend gateway to rewrite `FROM` statements on the fly. +```text +Error response from daemon: toomanyrequests: You have reached your pull rate limit. +You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit +``` ---- +Docker Hub [limits how many images you can pull](https://docs.docker.com/docker-hub/usage/), and CI runners that share an IP address hit that limit fast. The usual fix is to pull from a mirror such as Harbor, Nexus, Artifactory, or AWS ECR. That normally means rewriting every `FROM` line and every `docker pull`, or editing `daemon.json` as root. -## GitHub Actions +**fender does the redirect for you.** It's a small Go binary that sits between the Docker CLI and the Docker daemon and rewrites image names on the fly. You keep typing `docker pull nginx`, and the image comes from your mirror. -```yaml -steps: - - uses: fender-proxy/fender@v1 - with: - default-registry: registry.example.com +```bash +fender --default-registry registry.example.com - - run: docker pull nginx # → registry.example.com/library/nginx +docker pull nginx:latest # actually pulls registry.example.com/library/nginx:latest ``` -No `DOCKER_HOST` export needed — fender registers itself as the active Docker -context automatically. +When fender stops, your Docker setup goes back to exactly how it was. -### Inputs +## Contents -| Input | Description | Default | -|---|---|---| -| `version` | fender release tag | `latest` | -| `default-registry` | Registry for unqualified images | — | -| `registry-map` | Newline-separated `source: target` remappings | — | -| `auths` | Newline-separated registry credentials | — | -| `log-level` | `debug\|info\|warn\|error` | `info` | +- [Why fender?](#why-fender) +- [Quick start](#quick-start) +- [GitHub Actions](#github-actions) +- [GitLab CI](#gitlab-ci) +- [Configuration](#configuration) +- [Registry authentication](#registry-authentication) +- [Rewriting rules](#rewriting-rules) +- [How it works](#how-it-works) +- [FAQ](#faq) +- [Development](#development) -### Outputs - -| Output | Description | -|---|---| -| `socket` | Absolute path to the fender Unix socket | -| `version` | The fender version that was installed | +--- -### Example: private registry mirror +## Why fender? -```yaml -- uses: fender-proxy/fender@v1 - with: - registry-map: | - docker.io: nexus.corp/dockerhub-proxy - ghcr.io: nexus.corp/ghcr-proxy -``` +- **Stops `429 Too Many Requests` errors.** Route Docker Hub pulls through an internal mirror or pull-through cache: Harbor, Sonatype Nexus, JFrog Artifactory, AWS ECR pull-through cache, GCP Artifact Registry, or the GitLab Dependency Proxy. +- **No root access and no `daemon.json`.** Docker's built-in `registry-mirrors` setting requires editing `/etc/docker/daemon.json` and restarting `dockerd`. You can't do that on GitHub-hosted runners or locked-down machines. fender runs entirely in user space. +- **Mirrors any registry, not just Docker Hub.** Docker's built-in mirror only works for `docker.io`. fender can also redirect `ghcr.io`, `quay.io`, or any other registry. +- **No code changes.** Developers keep writing `FROM python:3.11-slim` and `docker pull nginx`. You don't have to edit hundreds of repositories across teams. +- **Works with `docker build`.** fender rewrites `FROM` lines in Dockerfiles, including BuildKit builds. ---- +### How it compares -## GitLab CI +| | fender | `registry-mirrors` in `daemon.json` | Rewriting image names by hand | +|---|:---:|:---:|:---:| +| No root or daemon restart | ✅ | ❌ | ✅ | +| Works on GitHub-hosted runners | ✅ | ❌ | ✅ | +| Mirrors registries other than Docker Hub | ✅ | ❌ | ✅ | +| No Dockerfile or script changes | ✅ | ✅ | ❌ | +| Rewrites `FROM` in `docker build` | ✅ | ✅ | ❌ | +| Injects mirror credentials automatically | ✅ | ❌ | ❌ | -```yaml -include: - - component: gitlab.com/fender-proxy/fender/fender@~latest - inputs: - default-registry: registry.example.com +--- -build: - extends: .fender - script: - - docker pull nginx # → registry.example.com/library/nginx -``` +## Quick start -`DOCKER_HOST` is automatically set in the job — no manual configuration needed. +fender runs on **Linux and macOS** (amd64 and arm64). -### Inputs +### 1. Install -| Input | Description | Default | -|---|---|---| -| `version` | fender release tag | `latest` | -| `default-registry` | Registry for unqualified images | — | -| `registry-map` | Newline-separated `source: target` remappings | — | -| `auths` | Newline-separated registry credentials | — | -| `log-level` | `debug\|info\|warn\|error` | `info` | +**Download a prebuilt binary** from the [latest release](https://github.com/fender-proxy/fender/releases/latest): ---- +```bash +# Pick one: linux_amd64, linux_arm64, darwin_amd64, darwin_arm64 +curl -fsSL https://github.com/fender-proxy/fender/releases/latest/download/fender_linux_amd64.tar.gz \ + | tar -xz fender +sudo mv fender /usr/local/bin/ +``` -**Requires Go 1.21+** +**Or install with Go** (requires the Go version listed in [`go.mod`](go.mod)): ```bash go install github.com/fender-proxy/fender@latest ``` -Or from source: +**Or build from source:** ```bash git clone https://github.com/fender-proxy/fender @@ -120,19 +102,17 @@ cd fender make install # → $GOPATH/bin/fender ``` ---- - -## Quick start +### 2. Run ```bash fender --default-registry registry.example.com ``` -That's it. fender will: +On startup, fender: -1. Detect your active Docker context and use its socket as the upstream -2. Create a `"fender"` Docker context pointing to its own socket -3. Set `"fender"` as the active context +1. Detects your active Docker context and uses its socket as the upstream. +2. Creates a Docker context called `fender` that points to its own socket. +3. Makes `fender` the active context. ``` time=… level=INFO msg="fender ready" @@ -144,7 +124,7 @@ time=… level=INFO msg="fender ready" ✓ Docker context "fender" is now active — no DOCKER_HOST export needed. ``` -All Docker tooling now routes through fender. No shell exports, no config changes. +### 3. Use Docker as usual ```bash docker pull nginx:latest # → registry.example.com/library/nginx:latest @@ -152,48 +132,93 @@ docker run ubuntu:22.04 id # → registry.example.com/library/ubuntu:22.04 docker pull ghcr.io/org/app # → unchanged (explicit registry) ``` -**On shutdown** (Ctrl-C or SIGTERM), fender removes the `"fender"` context and restores your previous context automatically. +When you stop fender (Ctrl-C or SIGTERM), it removes the `fender` context and switches you back to your previous one. --- -## Context awareness +## GitHub Actions + +Add one step before anything that uses Docker: -fender reads the active Docker context the same way the Docker CLI does: +```yaml +steps: + - uses: fender-proxy/fender@v0.3.2 + with: + default-registry: registry.example.com + - run: docker pull nginx # → registry.example.com/library/nginx ``` -DOCKER_HOST env var - → ~/.docker/config.json (currentContext field) - → ~/.docker/contexts/meta//meta.json - → platform default (/var/run/docker.sock or ~/.docker/run/docker.sock) + +You don't need to export `DOCKER_HOST`. fender makes itself the active Docker context, so later steps pick it up automatically. + +### Inputs + +| Input | Description | Default | +|---|---|---| +| `version` | fender release tag | `latest` | +| `default-registry` | Registry for unqualified images | — | +| `registry-map` | Newline-separated `source: target` remappings | — | +| `auths` | Newline-separated registry credentials | — | +| `log-level` | `debug\|info\|warn\|error` | `info` | + +### Outputs + +| Output | Description | +|---|---| +| `socket` | Absolute path to the fender Unix socket | +| `version` | The fender version that was installed | + +### Example: mirror Docker Hub and GHCR through Nexus + +```yaml +- uses: fender-proxy/fender@v0.3.2 + with: + registry-map: | + docker.io: nexus.corp/dockerhub-proxy + ghcr.io: nexus.corp/ghcr-proxy ``` -It also **watches** `~/.docker/` with `fsnotify`. If you switch contexts while fender is running, fender detects the change and updates its upstream socket live — no restart needed. +--- + +## GitLab CI -```bash -# fender is running… -docker context use my-other-context +fender ships as a [GitLab CI/CD component](https://docs.gitlab.com/ee/ci/components/): -# fender logs: -# level=INFO msg="Docker context changed — updating upstream" -# source="Docker context \"my-other-context\"" -# new_socket=/path/to/other.sock +```yaml +include: + - component: gitlab.com/fender-proxy/fender/fender@~latest + inputs: + default-registry: registry.example.com + +build: + extends: .fender + script: + - docker pull nginx # → registry.example.com/library/nginx ``` -### Crash recovery +`DOCKER_HOST` is set in the job automatically. -If fender exits without cleaning up (e.g. power loss, `kill -9`), it leaves a `"fender"` context behind. On the next run, fender detects the stale context, reads the `PreviousContext` stored in its metadata, and recovers cleanly — no manual intervention needed. +### Inputs + +| Input | Description | Default | +|---|---|---| +| `version` | fender release tag | `latest` | +| `default-registry` | Registry for unqualified images | — | +| `registry-map` | Newline-separated `source: target` remappings | — | +| `auths` | Newline-separated registry credentials | — | +| `log-level` | `debug\|info\|warn\|error` | `info` | --- ## Configuration -Configuration is loaded in this order (highest priority first): +fender works with zero config. Settings are applied in this order (highest priority first): ``` CLI flags > FENDER_* env vars > ~/.fender/config.yaml > defaults ``` -The config file is **optional** — fender works with zero config. To customise: +To start from the example config: ```bash mkdir -p ~/.fender @@ -250,14 +275,13 @@ log_level: "info" --- -## Registry Authentication +## Registry authentication -When images are rewritten to a different registry, they may require authentication credentials. `fender` automatically intercepts these calls and replaces/injects the `X-Registry-Auth` header with credentials matching the destination registry host. +If your mirror needs credentials, fender adds them for you. When it rewrites an image to a different registry, it sets the `X-Registry-Auth` header to the credentials for the destination registry. -You can configure authentication credentials in three ways: +There are three ways to configure credentials. -### 1. Standalone Auths block (Recommended) -Add an `auths` block in your `config.yaml` file: +### 1. An `auths` block (recommended) ```yaml auths: @@ -266,8 +290,7 @@ auths: password: mypassword ``` -### 2. Inline Registry Credentials -You can define credentials inline inside `default_registry` or `registry_map` mappings: +### 2. Inline with the registry ```yaml default_registry: @@ -282,12 +305,12 @@ registry_map: password: mypassword ``` -### 3. CI/CD Integrations -Pass credentials using CI secrets in your GitHub Actions workflow or GitLab CI pipeline: +### 3. From CI secrets **GitHub Actions:** + ```yaml -- uses: fender-proxy/fender@v1 +- uses: fender-proxy/fender@v0.3.2 with: default-registry: registry.example.com auths: | @@ -297,6 +320,7 @@ Pass credentials using CI secrets in your GitHub Actions workflow or GitLab CI p ``` **GitLab CI:** + ```yaml include: - component: gitlab.com/fender-proxy/fender/fender@~latest @@ -314,7 +338,7 @@ include: ### `default_registry` -Rewrites images that have no explicit registry **and** images that the Docker CLI has already normalised to `docker.io`. Both are redirected to `default_registry`: +Redirects images that have no explicit registry. The Docker CLI turns bare names like `nginx` into `docker.io/library/nginx` before sending them, so fender treats `docker.io` images the same way: | What you type | What Docker CLI sends | What fender forwards | |---|---|---| @@ -324,7 +348,7 @@ Rewrites images that have no explicit registry **and** images that the Docker CL ### `registry_map` -Replaces specific source registries. Can be used together with or instead of `default_registry`: +Redirects specific source registries. You can use it together with `default_registry` or on its own: ```yaml registry_map: @@ -340,25 +364,26 @@ registry_map: --- -## Docker API endpoints intercepted +## How it works -| Endpoint | What's rewritten | -|---|---| -| `POST /v*/containers/create` | `Image` field in JSON body (`docker run`) | -| `POST /v*/images/create` | `fromImage` query param (`docker pull`) | -| `GET /v*/images/{name}/json` | `{name}` path segment | -| `DELETE /v*/images/{name}` | `{name}` path segment | -| `POST /v*/images/{name}/push` | `{name}` path segment | -| `GET /v*/images/{name}/history` | `{name}` path segment | -| `POST /v*/images/{name}/tag` | `{name}` path segment | -| `/moby.buildkit.v1.Control/Solve` | BuildKit `SolveRequest` frontend source and options (`docker build`) | -| Everything else | Pass-through, byte-for-byte (streaming preserved) | +``` +docker pull nginx:latest # you type this + │ + ▼ Docker context: "fender" (~/.fender/fender.sock) + ┌─────────────────────────────────────────────────────────┐ + │ fender │ + │ docker.io/library/nginx:latest │ + │ ↓ rewrite │ + │ registry.example.com/library/nginx:latest │ + └─────────────────────────────────────────────────────────┘ + │ + ▼ upstream: active Docker context before fender started + Docker Daemon +``` -> **`docker build` and `FROM` lines:** `FROM` directives in a Dockerfile are fully intercepted and rewritten, even when using BuildKit (`DOCKER_BUILDKIT=1`). This works transparently by intercepting the gRPC `Solve` API call and injecting a custom, embedded BuildKit gateway frontend (`fender-frontend:local`) that rewrites base image references in the Dockerfile before invoking the standard compiler. +fender is a reverse proxy on a Unix socket. It registers itself as a Docker context, so the Docker CLI sends its API calls to fender. fender rewrites image names in the requests it cares about and passes everything else through unchanged. ---- - -## How it works +### Startup and per-request flow ``` ┌──────────────────────────────────────────────────────────────┐ @@ -386,11 +411,86 @@ registry_map: Docker Daemon ``` -fender uses Go's `httputil.ReverseProxy` over a Unix socket transport. The upstream socket is stored behind a `sync.RWMutex`, allowing `UpdateUpstream` to swap it live when the context watcher fires — with no connection drops for in-flight requests. +fender uses Go's `httputil.ReverseProxy` over a Unix socket transport. The upstream socket is stored behind a `sync.RWMutex`, so `UpdateUpstream` can swap it live when the context watcher fires without dropping in-flight requests. + +### Docker API endpoints intercepted + +| Endpoint | What's rewritten | +|---|---| +| `POST /v*/containers/create` | `Image` field in JSON body (`docker run`) | +| `POST /v*/images/create` | `fromImage` query param (`docker pull`) | +| `GET /v*/images/{name}/json` | `{name}` path segment | +| `DELETE /v*/images/{name}` | `{name}` path segment | +| `POST /v*/images/{name}/push` | `{name}` path segment | +| `GET /v*/images/{name}/history` | `{name}` path segment | +| `POST /v*/images/{name}/tag` | `{name}` path segment | +| `/moby.buildkit.v1.Control/Solve` | BuildKit `SolveRequest` frontend source and options (`docker build`) | +| Everything else | Pass-through, byte-for-byte (streaming preserved) | + +> **`docker build` and `FROM` lines:** fender rewrites `FROM` lines even with BuildKit (`DOCKER_BUILDKIT=1`). It intercepts the gRPC `Solve` call and swaps in an embedded BuildKit frontend (`fender-frontend:local`). That frontend rewrites base image references in the Dockerfile, then hands off to the standard Dockerfile compiler. + +### Context awareness + +fender finds the active Docker context the same way the Docker CLI does: + +``` +DOCKER_HOST env var + → ~/.docker/config.json (currentContext field) + → ~/.docker/contexts/meta//meta.json + → platform default (/var/run/docker.sock or ~/.docker/run/docker.sock) +``` + +It also watches `~/.docker/` with `fsnotify`. If you switch contexts while fender is running, fender picks up the new socket immediately, with no restart: + +```bash +# fender is running… +docker context use my-other-context + +# fender logs: +# level=INFO msg="Docker context changed — updating upstream" +# source="Docker context \"my-other-context\"" +# new_socket=/path/to/other.sock +``` + +### Crash recovery + +If fender exits without cleaning up (for example after a power loss or `kill -9`), it leaves a `fender` context behind. On the next run, fender finds that stale context, reads the previous context it saved in the context metadata, and recovers on its own. --- -## Makefile +## FAQ + +### How do I fix "toomanyrequests: You have reached your pull rate limit" in CI? + +Point Docker at a registry mirror or pull-through cache instead of Docker Hub. With fender, add the [GitHub Action](#github-actions) or [GitLab component](#gitlab-ci) and set `default-registry` to your mirror. Existing `docker pull`, `docker run`, and `FROM` lines then pull from the mirror without any other changes. + +### How is this different from Docker's `registry-mirrors` setting? + +`registry-mirrors` lives in `/etc/docker/daemon.json`, needs root, and requires restarting the Docker daemon. It also only mirrors Docker Hub. fender runs as a normal user, needs no restart, works on hosted CI runners, and can redirect any registry, including `ghcr.io` and `quay.io`. + +### Do I need to change my Dockerfiles or CI scripts? + +No. fender rewrites image names as they pass through, so `FROM nginx` and `docker pull nginx` keep working as written. + +### Does it work with `docker build`? + +Yes, including BuildKit. fender rewrites `FROM` lines during the build. See [How it works](#how-it-works) for details. + +### What happens to my Docker setup when fender stops? + +fender removes its `fender` context and switches you back to whatever context you had before. If it crashes, it cleans up on the next run. + +### Which registries can I use as a mirror? + +Any registry that speaks the Docker Registry HTTP API, including Harbor, Sonatype Nexus, JFrog Artifactory, AWS ECR, GCP Artifact Registry, and the GitLab Dependency Proxy. + +### Does fender work on Windows? + +Not yet. fender uses Unix sockets and currently supports Linux and macOS. + +--- + +## Development ```bash make build # → ./bin/fender @@ -400,8 +500,12 @@ make test # run unit tests make clean # remove ./bin ``` ---- +## Contributing + +Bug reports, feature requests, and pull requests are welcome. Please [open an issue](https://github.com/fender-proxy/fender/issues) to discuss larger changes first. + +If fender saves you from a rate-limited pipeline, a ⭐ on the repo helps other people find it. ## License -MIT +[MIT](LICENSE)