Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,18 @@ cargo nextest run -p <crate> when suggesting fast local test runs

## Key Components (Modules)

This is a monorepo containing many components. Some key ones include:

* **`mono` / `ceres` / `jupiter` / `moon`**: These are various services and libraries within the monorepo, primarily written in Rust and TypeScript.
* **`orion`**: Build orchestration and workspace management.
* **`saturn`**: Policy and permission management.

**Note**: `scorpio` (FUSE filesystem) has been moved to its own repository at [scorpiofs](https://github.com/web3infra-foundation/scorpiofs).
| Crate | Role |
|-------|------|
| `mono` | Server binary (HTTP REST, Git HTTP/SSH, Swagger) |
| `ceres` | Domain library: transport, application, HTTP DTOs |
| `jupiter` / `callisto` | Storage and SeaORM entities |
| `jupiter-migrate` | Database migrations |
| `orion` / `orion-server` / `orion-scheduler` | Build orchestration |
| `saturn` | Cedar policy |
| `vault` | Crypto and secrets |
| `moon` | Frontend (TypeScript) |

See [docs/architecture.md](docs/architecture.md) for the full workspace map.

## Coding style & quality

Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/base.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,45 @@ jobs:
with:
shared-key: base-${{ runner.os }}-${{ hashFiles('**/Cargo.lock') }}
cache-on-failure: true
- name: Enforce DTO boundary (mono must not import jupiter::model)
run: |
if rg -q 'jupiter::model' mono/; then
echo "mono must not depend on jupiter::model; use ceres::model and MonoApiService facades"
rg 'jupiter::model' mono/
exit 1
fi
- name: Enforce module dependency boundaries
run: |
if rg -q 'use callisto::' mono/src/; then
echo "mono/src must not import callisto directly; use ceres::model and MonoApiService facades"
rg 'use callisto::' mono/src/
exit 1
fi
if rg -q 'jupiter::service::' mono/src/; then
echo "mono/src must not import jupiter::service directly; use MonoApiService facades"
rg 'jupiter::service::' mono/src/
exit 1
fi
if rg -q 'api_service::mono::MonoApiService' ceres/src/transport/; then
echo "ceres transport must not depend on MonoApiService"
rg 'api_service::mono::MonoApiService' ceres/src/transport/
exit 1
fi
if rg -q 'use crate::(pack|transport)' ceres/src/application/; then
echo "ceres application must not import transport/pack directly"
rg 'use crate::(pack|transport)' ceres/src/application/
exit 1
fi
if rg --glob '!lfs_router.rs' -q '\.storage\.|_stg\(\)' mono/src/api/router/; then
echo "mono routers must use MonoApiService facades, not storage"
rg --glob '!lfs_router.rs' '\.storage\.|_stg\(\)' mono/src/api/router/
exit 1
fi
if rg -q 'ceres::(api_service|pack|protocol|build_trigger|code_edit)' mono/; then
echo "mono must use ceres::application::* and ceres::transport::* instead of legacy re-exports"
rg 'ceres::(api_service|pack|protocol|build_trigger|code_edit)' mono/
exit 1
fi
- name: Run cargo clippy
run: |
sccache --start-server || true
Expand Down
32 changes: 18 additions & 14 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 2 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ members = [
"ceres",
"clients/orion-client",
"common",
"context",
"io-orbit",
"jupiter",
"jupiter/callisto",
"jupiter-migrate",
"mono",
"orion",
"orion-scheduler",
Expand All @@ -19,7 +19,6 @@ members = [
"vault",
]
default-members = ["mono", "orion", "orion-server", "orion-scheduler"]
exclude = ["tools/artifacts-compose-e2e"]
resolver = "3"


Expand All @@ -29,15 +28,14 @@ io-orbit = { path = "io-orbit" }
mono = { path = "mono" }
common = { path = "common" }
jupiter = { path = "jupiter" }
jupiter-migrate = { path = "jupiter-migrate" }
ceres = { path = "ceres" }
callisto = { path = "jupiter/callisto" }
vault = { path = "vault" }
saturn = { path = "saturn" }
orion = { path = "orion" }
orion-client = { path = "clients/orion-client" }

context = { path = "context" }

git-internal = "0.7.6"
libvault-core = "0.1.0"

Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ Mega is evolving toward deeper AI-native capabilities:

To facilitate a rapid deployment and hands-on experience with the Mega service, the following instructions are derived from the project's [documentation](https://github.com/web3infra-foundation/mega/tree/main/docker).

- **Docker demo (recommended):** [docker/README.md](docker/README.md)
- **Native development:** [docs/development.md](docs/development.md)
- **Architecture:** [docs/architecture.md](docs/architecture.md)

Related projects: [Libra](https://github.com/web3infra-foundation/libra) (Git-compatible agent client), [ScorpioFS](https://github.com/web3infra-foundation/scorpiofs) (FUSE monorepo mount).

## Community

Discord Channel - https://discord.gg/HMFuu6pJmQ
Expand Down
7 changes: 6 additions & 1 deletion ceres/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,18 @@ uuid = { workspace = true, features = ["v4"] }
reqwest = { workspace = true, features = ["json", "stream"] }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "net", "process"] }
tokio-stream = { workspace = true }
axum = { workspace = true }
axum-core = "0.5.6"
async-recursion = { workspace = true }
sysinfo = { workspace = true }
utoipa = { workspace = true }
regex = { workspace = true }
tokio-util = { workspace = true }
rkyv = { workspace = true }

[features]
migrate = ["jupiter/migrate"]

[dev-dependencies]
axum = { workspace = true }
jupiter-migrate = { workspace = true }
tempfile = { workspace = true }
42 changes: 40 additions & 2 deletions ceres/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Monorepo domain library for Mega: Git transport, REST application logic, and sha
ceres/src/
├── lib.rs
├── bus/ # Transport ↔ application event bus
├── infra/ # Shared infrastructure (GitObjectCache)
├── infra/ # Shared infrastructure (GitObjectCache, pack streams, decode errors)
├── transport/
│ ├── protocol/ # Smart HTTP/SSH Git protocol
│ └── pack/ # receive-pack / upload-pack handlers
Expand All @@ -20,7 +20,9 @@ ceres/src/
├── diff/, merge_checker/, lfs/
```

Legacy paths (`ceres::protocol`, `ceres::pack`, `ceres::api_service`, etc.) remain as re-exports in `lib.rs` for compatibility with `mono`.
`mono` uses `ceres::application::*` and `ceres::transport::*` for domain logic and Git transport.

`axum-core` is confined to `ceres/infra/pack_decode.rs` for `git-internal` pack decode stream errors until upstream accepts `std::io::Error`.

## Dependency rules

Expand All @@ -31,6 +33,42 @@ Legacy paths (`ceres::protocol`, `ceres::pack`, `ceres::api_service`, etc.) rema
| `bus` | Minimal shared types for events | `transport` / `application` implementations |
| `mono` (binary) | Assembles `TransportRuntime` + injects handlers | — |

## Model boundary

Three DTO layers; keep imports aligned with this table:

| Layer | Crate / path | Role | Consumers |
|-------|--------------|------|-----------|
| Wire | `api-model` | mono ↔ orion cross-process protocol (buck2, artifacts, shared pagination wrappers) | `mono`, `orion`, `orion-client`, `ceres` (pagination only where needed) |
| HTTP / OpenAPI | `ceres/model` | All mono REST request/response types + `utoipa` schemas | `mono` routers, `ceres` application |
| Storage assembly | `jupiter/model` | Bundles of `callisto` entities from storage/services; no serde/utoipa | `jupiter` storage/service, `ceres` application only |

Rules:

- `mono` routers must **not** `use jupiter::model` — map via `ceres::model` and `MonoApiService` facades.
- `mono/src` must **not** `use callisto::` or `jupiter::service::` — storage entities and service calls stay in `ceres` application layer.
- `ceres/src/transport` must **not** reference `MonoApiService` (transport ↔ application boundary).
- `api-model` is **not** mono HTTP schema (except shared wrappers like `CommonPage` / `Pagination`).
- `ceres/model` is the mapping hub: `impl From<jupiter::model::*>` and `impl From<callisto::*>` live here.
- `application/build_trigger/model` is a ceres subdomain API schema (build triggers); same HTTP rules as `ceres/model`, kept alongside orchestration until a later consolidation.

## Error type boundaries

| Layer | Error type | HTTP adapter (mono) |
|-------|------------|---------------------|
| `ceres/application`, `jupiter` | `MegaError` | `ApiError` (`mono/src/api/error.rs`) |
| `ceres/transport`, Git Smart HTTP/SSH | `ProtocolError` | `protocol_error::into_response` |
| Buck upload | `BuckError` (via `MegaError::Buck`) | `ApiError` |
| Git LFS | `GitLFSError` | `map_lfs_error` in `lfs_router` |

Rules:

- REST routers and `MonoApiService` use `MegaError` only; `api_handler` resolves import vs mono handlers and returns `MegaError`.
- `ProtocolError` is confined to Git client protocol paths; use `mega_to_protocol_error` at transport boundaries when mapping domain failures.
- Do not use `ProtocolError` in REST handlers.

Long term: extract `ceres/model` → `mega-api-types` only if a non-mono consumer needs HTTP DTOs without ceres domain code.

## Git push event flow

```mermaid
Expand Down
5 changes: 3 additions & 2 deletions ceres/src/application/api_service/blame_ops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ use git_internal::{
};

use crate::{
api_service::ApiHandler,
application::api_service::ApiHandler,
model::blame::{BlameBlock, BlameInfo, BlameQuery, BlameResult, Contributor},
};

Expand Down Expand Up @@ -182,7 +182,8 @@ pub async fn get_file_blame<T: ApiHandler + ?Sized>(

// Resolve starting commit from refs
let start_commit =
crate::api_service::commit_ops::resolve_start_commit(handler, ref_name).await?;
crate::application::api_service::commit_ops::resolve_start_commit(handler, ref_name)
.await?;

// Get file content and blob hash at start commit
let (current_content, start_blob_hash) =
Expand Down
2 changes: 1 addition & 1 deletion ceres/src/application/api_service/blob_ops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use git_internal::{
use sha1::{Digest, Sha1};

use crate::{
api_service::{ApiHandler, tree_ops},
application::api_service::{ApiHandler, tree_ops},
model::git::DiffPreviewPayload,
};

Expand Down
2 changes: 1 addition & 1 deletion ceres/src/application/api_service/commit_ops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ use jupiter::redis::AsyncCommands;
use serde::{Deserialize, Serialize};

use crate::{
api_service::{ApiHandler, history, tree_ops},
application::api_service::{ApiHandler, history, tree_ops},
model::{
change_list::{DiffItemSchema, MuiTreeNode},
commit::{CommitFilesChangedPage, CommitSummary, GpgStatus},
Expand Down
5 changes: 3 additions & 2 deletions ceres/src/application/api_service/history.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ use git_internal::{
},
};

use crate::api_service::ApiHandler;
use crate::application::api_service::ApiHandler;

const MAX_ITERATIONS: usize = 10_000;

Expand Down Expand Up @@ -67,7 +67,8 @@ pub async fn item_to_commit_map<T: ApiHandler + ?Sized>(
) -> Result<HashMap<TreeItem, Option<Commit>>, GitError> {
// Resolve the starting commit
let start_commit_arc =
crate::api_service::commit_ops::resolve_start_commit(handler, reference).await?;
crate::application::api_service::commit_ops::resolve_start_commit(handler, reference)
.await?;

// Get the tree at the specified path from the resolved start commit
let start_tree = handler
Expand Down
Loading
Loading