Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions docs/architecture/health_aggregation.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,19 +290,24 @@ In certain conditions the scraping process will place a health alert on the host

## Health report overrides

Site administrators are able to update the health state of any NICo managed host via
the API calls `InsertHealthReportOverride` and `RemoveHealthReportOverride`.
Site administrators can inspect and update the health state of a NICo-managed host
through these REST operations:

- `GET /v2/org/{org}/nico/machine/{machineId}/health-report` lists all reports.
- `PUT /v2/org/{org}/nico/machine/{machineId}/health-report` creates or updates an override.
- `DELETE /v2/org/{org}/nico/machine/{machineId}/health-report/{source}` removes an override.

The override API offers 2 different modes of operation:
1. `merge` (default) - In this mode, any health probe alerts indicated in the override
will get merged with health probe alerts reported by builtin NICo tools in order

1. `Merge` - In this mode, any health probe alerts indicated in the override
will get merged with health probe alerts reported by built-in NICo tools in order
to derive the aggregate host health status. **This mode is meant to augment the internal health monitoring mechanism with additional sources of health data**
1. `replace` - In this mode, the health probe alerts reported by builtin NICo
2. `Replace` - In this mode, the health probe alerts reported by built-in NICo
monitoring tools will be ignored. Only alerts that are passed as part of the
override will be taken into account. If the override list is empty, the system
will behave as if the Host would be fully healthy. **This mode is meant to bypass the internal health data in case the site operator desires a different behavior**

The API allows to apply multiple `merge` overrides to a hosts health at the same time by using a different `HealthReport::source` identifier.
The API allows multiple `Merge` overrides on a host at the same time by using a different `source` identifier for each report.
This allows to integrate health information from multiple external systems and users which are not at risk of overriding each others data. E.g. health information from an external fleet health monitoring system and from SREs can be stored independently.

If a ManagedHost's health is overridden, the remaining behavior is exactly the same
Expand Down
6 changes: 3 additions & 3 deletions docs/configuration/org-permissions.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,9 @@ Example response for a human user:

Service accounts have empty `email`/`firstName`/`lastName`. Human users have those populated from the IdP.

This endpoint does not return role information directly -- roles are in the token and validated server-side. Confirm which roles you hold by attempting role-gated operations: `nicocli allocation list` requires Provider Admin; `nicocli tenant get-current-tenant` requires Tenant Admin.
This endpoint does not return role information directly -- roles are in the token and validated server-side. Confirm which roles you hold by attempting role-gated operations: `nicocli allocation list` requires Provider Admin; `nicocli tenant current` requires Tenant Admin.

If your deployment is configured for service-account auth, use `nicocli service-account get` to retrieve the current org's service-account status, including the auto-created provider and tenant IDs.
If your deployment is configured for service-account auth, use `nicocli service-account current` to retrieve the current org's service-account status, including the auto-created provider and tenant IDs.

## Listing Tenant Members

Expand All @@ -77,5 +77,5 @@ Remove the role assignment (or org membership) in the identity provider. The cha
1. Create the IdP organization (or group) for the tenant.
2. Invite at least one user with `TENANT_ADMIN` so they can provision the tenant.
3. Invite additional team members with appropriate roles.
4. Verify from the tenant side: `nicocli user get` and `nicocli tenant get-current-tenant`.
4. Verify from the tenant side: `nicocli user get` and `nicocli tenant current`.
5. On the provider side, ensure at least one user holds `PROVIDER_ADMIN`.
28 changes: 14 additions & 14 deletions docs/configuration/tenant_management.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ nicocli user get
NICo uses a lazy creation model for tenants. There is no explicit "create tenant" API call. Instead, a tenant record is automatically created the first time a Tenant Admin retrieves the current tenant for their organization:

```
nicocli tenant get-current-tenant
nicocli tenant current
```

If no tenant exists for the configured org, NICo creates one and returns it. If a tenant already exists, the same command returns the existing record. The operation is idempotent.
Expand All @@ -78,7 +78,7 @@ If either condition is not met, the API returns HTTP 403. NICo trusts whatever t
### Worked Example

```
$ nicocli tenant get-current-tenant
$ nicocli tenant current
{
"capabilities": {
"targetedInstanceCreation": true
Expand All @@ -96,14 +96,14 @@ $ nicocli tenant get-current-tenant
| `id` | UUID identifier for the tenant, used in all subsequent API calls |
| `org` | Organization name (matches your config `api.org`) |
| `orgDisplayName` | Human-readable name pulled from the IdP's org metadata |
| `capabilities.targetedInstanceCreation` | Whether this tenant can specify a particular machine ID when creating instances. Set during initial tenant creation: lazy-create via `tenant get-current-tenant` typically leaves it `false`; the service-account bootstrap path (`service-account get`) sets it `true` for self-tenants. |
| `capabilities.targetedInstanceCreation` | Whether this tenant can specify a particular machine ID when creating instances. Set during initial tenant creation: lazy-create via `tenant current` typically leaves it `false`; the service-account bootstrap path (`service-account current`) sets it `true` for self-tenants. |

### Verifying the Tenant

Check tenant health with the stats endpoint:

```
nicocli tenant get-current-tenant-stats
```text
nicocli tenant stats
```

Example response for a tenant in active use:
Expand Down Expand Up @@ -401,7 +401,7 @@ A tenant can have multiple allocations at the same site with different resource

### Allocation Workflow Summary

1. **Provision the tenant** -- `nicocli tenant get-current-tenant`
1. **Provision the tenant** -- `nicocli tenant current`
2. **Establish a tenant account** -- Provider admin links provider to tenant org
3. **Discover available resources** -- List sites, instance types, and IP blocks
4. **Create compute allocation(s)** -- One per instance type the tenant needs
Expand Down Expand Up @@ -703,7 +703,7 @@ See the [Machine Reboot](../playbooks/machine_reboot.md) and [Force Delete](../p
### Viewing the Current Tenant

```
nicocli tenant get-current-tenant
nicocli tenant current
```

For provider admins needing visibility across tenants, list tenant accounts (a filter flag is required):
Expand All @@ -715,7 +715,7 @@ nicocli tenant-account list --infrastructure-provider-id <provider-uuid> --outpu
### Monitoring Tenant Health

```
nicocli tenant get-current-tenant-stats
nicocli tenant stats
```

Non-zero `error` counts warrant investigation:
Expand Down Expand Up @@ -762,7 +762,7 @@ This section ties together the full Day One workflow. The TUI flow is the recomm
### Step 1: Provision the Tenant (Tenant Admin)

```
nicocli tenant get-current-tenant
nicocli tenant current
```

Idempotent -- creates the tenant lazily on first call.
Expand Down Expand Up @@ -838,7 +838,7 @@ For automation, use `--data-file` -- see the Launching an Instance section above
### Step 10: Verify

```
nicocli tenant get-current-tenant-stats
nicocli tenant stats
nicocli instance list --output table
nicocli instance status-history <instance-id>
```
Expand Down Expand Up @@ -869,7 +869,7 @@ The instance should reach `Ready` (or `BootCompleted` if `phoneHomeEnabled: true
Use `--debug` on any command to see the full HTTP request and response. The token is redacted in the log; the path-rewriting from `nico` to whatever `api.name` is set to is visible. Real output:

```
$ nicocli --debug tenant get-current-tenant
$ nicocli --debug tenant current
time=... msg="API request: GET http://<api>/v2/org/<org>/<api-name>/tenant/current"
time=... msg="Request headers: {\"Accept\":[\"application/json\"],\"Authorization\":[\"Bearer <redacted>\"]}"
time=... msg="API response: ... -> 200 OK"
Expand Down Expand Up @@ -908,9 +908,9 @@ Flag-first ordering -- always put flags before positional args.
| Operation | Command | Role Required |
|-----------|---------|--------------|
| View current user | `nicocli user get` | Any authenticated user |
| View current tenant | `nicocli tenant get-current-tenant` | Tenant Admin |
| View tenant stats | `nicocli tenant get-current-tenant-stats` | Tenant Admin |
| Service-account status | `nicocli service-account get` | Any authenticated user |
| View current tenant | `nicocli tenant current` | Tenant Admin |
| View tenant stats | `nicocli tenant stats` | Tenant Admin |
| Service-account status | `nicocli service-account current` | Any authenticated user |
| List tenant accounts | `nicocli tenant-account list --tenant-id <id>` (or `--infrastructure-provider-id`) | Provider or Tenant Admin |
| Create tenant account | `nicocli tenant-account create --infrastructure-provider-id <id> --tenant-org <org>` | Provider Admin |
| Accept tenant account | `nicocli tenant-account update --data '{}' <account-id>` | Tenant Admin |
Expand Down
24 changes: 24 additions & 0 deletions docs/dpu-management/dpu-lifecycle-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,30 @@ nico-admin-cli -a <api-url> dpu reprovision set \

Firmware is always verified and updated during reprovisioning regardless of whether `--update-firmware` is passed. The `--update-firmware` flag is accepted but deprecated.

REST callers must create the same health precondition before starting reprovisioning:

```bash
curl -X PUT "${BASE_URL}/v2/org/${ORG}/nico/machine/${MACHINE_ID}/health-report" \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"source": "maintenance.dpu-reprovision",
"mode": "Merge",
"alerts": [{
"id": "HostUpdateInProgress",
"message": "DPU reprovisioning in progress",
"classifications": ["PreventAllocations"]
}]
}'

curl -X PATCH "${BASE_URL}/v2/org/${ORG}/nico/machine/${MACHINE_ID}/dpu/reprovision" \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{"mode":"Set"}'
```

Use `Set` to start reprovisioning and `Clear` to remove a pending request. `Restart` accepts a host ID only and restarts DPUs that already have a reprovisioning request. If an Instance is attached to the Machine, also pass `"acknowledgeAttachedInstance": true`. The REST `updateFirmware` field is accepted for compatibility, but firmware is always verified and updated during reprovisioning.

### Monitoring Reprovisioning Progress

```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/manuals/building_nico_containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ top of the repo with a single `make` command:

```sh
make images # deployable stack: NICo Core (nico) + the REST service images
make images-all # the above plus the machine-validation and x86 boot-artifact images
make images-all # the above plus machine-validation and x86_64/aarch64 boot-artifact images
```

Images are tagged `localhost:5000/<name>:latest` by default. Override the registry and
Expand Down
75 changes: 67 additions & 8 deletions docs/manuals/nicocli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ The [Day One Operations](day_one_operations.md) guide uses this reference for al

## Installation

Building `nicocli` or `nico-mcp` requires Go 1.26.4 or newer.

Build and install from the `rest-api/` directory of the `infra-controller` repo:

```
Expand Down Expand Up @@ -256,7 +258,7 @@ After login, confirm nicocli can reach the API and your identity is correct:
```
nicocli site list # any list returns -> auth works
nicocli user get # returns the caller's user record
nicocli service-account get # for service-account deployments only
nicocli service-account current # for service-account deployments only
```

`user get` returns the authenticated identity as NICo sees it. Service accounts have blank `email`/`firstName`/`lastName`; human users have those populated from the IdP.
Expand Down Expand Up @@ -289,14 +291,16 @@ Run `nicocli <resource> --help` to enumerate available actions and sub-resources

### Action name resolution

The CLI's action-name resolver collapses `get-current-X` operation IDs to a short `get` action when there's no sibling collision. When two operation IDs would collide on the short form, both keep their full names.
The CLI's action-name resolver maps `get-current-X` operations to `current`. Current-resource statistics map to `stats`, so these operations stay distinct without exposing their full OpenAPI operation IDs.

| Operation ID | Resource | Action name |
|--------------|----------|-------------|
| `get-current-user` | `user` | `get` (no collision) |
| `get-current-service-account` | `service-account` | `get` (no collision) |
| `get-current-tenant` | `tenant` | `get-current-tenant` (collision with `get-current-tenant-stats`) |
| `get-current-tenant-stats` | `tenant` | `get-current-tenant-stats` (collision) |
| `get-user` | `user` | `get` |
| `get-current-service-account` | `service-account` | `current` |
| `get-current-tenant` | `tenant` | `current` |
| `get-current-tenant-stats` | `tenant` | `stats` |
| `get-current-infrastructure-provider` | `infrastructure-provider` | `current` |
| `get-current-infrastructure-provider-stats` | `infrastructure-provider` | `stats` |

Use `--help` to confirm the actual action name for any resource if you're unsure.

Expand Down Expand Up @@ -359,7 +363,7 @@ List and get commands support `--output <format>`:

```
nicocli site list --output table
nicocli tenant get-current-tenant --output yaml
nicocli tenant current --output yaml
nicocli audit list --output json --page-size 50 | jq '.[] | select(.statusCode >= 400)'
```

Expand Down Expand Up @@ -402,7 +406,7 @@ The `--query` flag is a **free-text search across `name`, `description`, and `st
The global `--debug` flag logs the full HTTP request and response for the wrapped command. The bearer token is redacted; everything else is visible:

```
$ nicocli --debug tenant get-current-tenant
$ nicocli --debug tenant current
time=... msg="API request: GET https://nico.example.com/v2/org/my-org/nico/tenant/current"
time=... msg="Request headers: {\"Accept\":[\"application/json\"],\"Authorization\":[\"Bearer <redacted>\"]}"
time=... msg="API response: ... -> 200 OK"
Expand Down Expand Up @@ -446,6 +450,61 @@ Behavior:

The TUI's interactive forms for `create` commands prompt for fields in order with type-aware pickers. For first-time operators this is significantly easier than constructing JSON bodies by hand. For scripts and automation, fall back to the non-interactive command form.

## MCP Server

`nico-mcp` is a standalone server that exposes every `GET` operation in the embedded NICo OpenAPI specification as a Model Context Protocol tool over streamable HTTP. It is separate from `nicocli`; the `nicocli mcp` command prints build and run instructions but does not start the server.

Build and run it from the `rest-api` directory:

```bash
make nico-mcp
nico-mcp --listen :8080 --path /mcp \
--base-url https://nico.example.com --org tester
```

The server has these properties:

- Only OpenAPI `GET` operations are exposed. `POST`, `PATCH`, `PUT`, and `DELETE` operations are excluded.
- Tool names use `nico_<snake_case(operationId)>`, such as `nico_get_all_site`.
- The streamable HTTP handler is stateless and returns one `application/json` response for each request. It does not retain MCP session state or emit server-sent events.
- An inbound `Authorization: Bearer <token>` header is forwarded to NICo REST. NICo REST remains responsible for authentication and authorization.

### Server configuration

| Flag | Environment variable | Description |
|------|----------------------|-------------|
| `--listen` | `NICO_MCP_LISTEN` | Listen address. Defaults to `:8080`. |
| `--path` | `NICO_MCP_PATH` | MCP handler path. Defaults to `/mcp`. |
| `--shutdown-timeout` | `NICO_MCP_SHUTDOWN_TIMEOUT` | Graceful shutdown timeout. Defaults to `10s`. |
| `--base-url` | `NICO_BASE_URL` | Default NICo REST base URL. |
| `--org` | `NICO_ORG` | Default organization for REST requests. |
| `--api-name` | `NICO_API_NAME` | API path segment. Defaults to `nico`. |
| `--token` | `NICO_TOKEN` | Default bearer token. |
| `--debug` | None | Log full HTTP requests and responses. |

Every generated tool also accepts `org`, `base_url`, `api_name`, and `token` arguments. For each call, an explicit tool argument takes precedence over the inbound authorization header for `token`, followed by the server startup default. The server does not read `~/.nico/config.yaml`; `org` and `base_url` must resolve from a tool argument, startup flag, or environment variable.

List the generated tools:

```bash
curl -sS http://localhost:8080/mcp \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' | jq
```

Call a tool while passing the caller's bearer token through to NICo REST:

```bash
curl -sS http://localhost:8080/mcp \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-H "Authorization: Bearer $TOKEN" \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"nico_get_all_site","arguments":{}}}' | jq
```

The `helm/rest/nico-mcp` chart deploys the same server as a `ClusterIP` service. Set `global.image.repository` and `global.image.tag` for the `nico-mcp` image. The chart accepts optional `config.baseURL`, `config.org`, and `config.apiName` defaults; bearer tokens are intentionally supplied per request rather than through chart values.

## Quick Reference

| Operation | Command |
Expand Down
Loading
Loading