Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 7 additions & 9 deletions crates/chunk-build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,16 +70,15 @@ composes down the directory tree. An app's own backend functions live in its `se
`apps/<name>/app.toml` are still read and belong to the root scope; an app can't have both.

An app's `worlds` and the `packs` of apps and scopes name assets by `source`: a path inside the app's `assets/`, or for
a scope the project's `assets/`, with no symlinks. A world is a `.polar` file or an Anvil save directory, optionally
cropped with `chunks: { from: [x, z], to: [x, z] }`; a pack is a directory with `pack.mcmeta` or a `.zip`, optionally
`required` and with a `prompt`. Pack names are unique in the project, and an app's players hold the packs of its scopes,
outermost first, then its own. Gameplay code names worlds through the generated `Worlds`, such as `Worlds.Arena.KOTH`.
a scope the project's `assets/`, with no symlinks. A world is a `.polar` file; Chunk doesn't convert Anvil saves. A pack
is a directory with `pack.mcmeta` or a `.zip`, optionally `required` and with a `prompt`. Pack names are unique in the
project, and an app's players hold the packs of its scopes, outermost first, then its own. Gameplay code names worlds
through the generated `Worlds`, such as `Worlds.Arena.KOTH`.

Manifests are read without running any code: the default export must be a direct `defineApp({...})` or
`defineScope({...})` call, and IDs, runtime settings, implementation keys and destination identities must be literals.
`chunk inspect` prints what the tools read, as JSON with `version: 1`, `apps` and `local`; each app lists its `worlds`
with their project-relative `source`, `format` (`anvil` or `polar`) and `chunks`. `inspect`, `gen`, `build` and `dev`
need `chunk.toml`; `codegen` doesn't.
with their project-relative `source`. `inspect`, `gen`, `build` and `dev` need `chunk.toml`; `codegen` doesn't.

## Backend compilation

Expand Down Expand Up @@ -128,9 +127,8 @@ A release holds:

A release holds no assets. `chunk build` then builds the project's asset revision into `dist/assets/`
(`.chunk/local/assets/` for `chunk dev`): the files of `assets/` and each app's `assets/`, each app's worlds and every
pack, stored once by SHA-256 under `blobs/`, and the revision's manifest under `revisions/<id>.json`. Anvil worlds are
read as Gradle converted them to `.chunk/build/worlds/<app>/<name>.polar`, and pack directories are zipped reproducibly.
A deployment pairs a release with a revision holding every world and pack the release declares.
pack, stored once by SHA-256 under `blobs/`, and the revision's manifest under `revisions/<id>.json`. Pack directories
are zipped reproducibly. A deployment pairs a release with a revision holding every world and pack the release declares.

The release ID is one SHA-256 over the sorted payload names and bytes plus the normalized manifest, so identical inputs
give the same ID and archive bytes on any machine; archives use fixed ordering, permissions and timestamps. Local paths,
Expand Down
5 changes: 2 additions & 3 deletions crates/chunk-build/sdk/src/apps.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,10 @@ export interface PackOptions {
/** Plain text the client shows when it asks the player to accept the pack. */
readonly prompt?: string;
}
/** A world in the app's `assets/`: a `.polar` file, or an Anvil save directory the build converts to Polar. */
/** A world in the app's `assets/`. */
export interface WorldOptions {
/** A `.polar` file, relative to the app's `assets/`. */
readonly source: string;
/** The inclusive chunk coordinates an Anvil save is cropped to. */
readonly chunks?: { readonly from: readonly [number, number]; readonly to: readonly [number, number] };
}
export interface ScopeOptions {
readonly hooks?: Readonly<Record<string, AnyHook>>;
Expand Down
2 changes: 1 addition & 1 deletion crates/chunk-build/sdk/test/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -438,7 +438,7 @@ defineApp({
defineApp({ id: "lobby", destinations: { main: { implementation: "default", key: "lobby" } } });
defineApp({
id: "koth",
worlds: { hill: { source: "worlds/hill", chunks: { from: [-8, -8], to: [7, 7] } } },
worlds: { hill: { source: "worlds/hill.polar" } },
packs: { hud: { source: "packs/hud", required: true, prompt: "Needed for the HUD" } },
});
defineScope({ packs: { base: { source: "packs/base" } } });
Expand Down
25 changes: 6 additions & 19 deletions crates/chunk-build/src/assets/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,19 @@ use zip::{CompressionMethod, ZipArchive, ZipWriter, write::SimpleFileOptions};

use super::Store;
use crate::{
project::{self, Pack, WorldFormat},
project::{self, Pack},
publication::MAX_COMPONENTS,
};

/// The bytes every Polar world starts with.
const POLAR_MAGIC: &[u8; 4] = b"Polr";

/// Builds the asset revision of the project at `root` into `store` and writes it there: the files of `assets/` and of
/// each app's `assets/`, except declared world and pack sources; each app's worlds, Polar sources as they are and Anvil
/// saves as the Gradle build converted them; and every pack, directories zipped reproducibly.
/// each app's `assets/`, except declared world and pack sources; each app's Polar worlds; and every pack, directories
/// zipped reproducibly.
/// # Errors
/// Rejects invalid declarations, symlinks, nonportable paths, unconverted Anvil worlds, sources of the wrong format,
/// revisions over the limits of [`chunk_contract`], and filesystem failures.
/// Rejects invalid declarations, symlinks, nonportable paths, sources of the wrong format, revisions over the limits of
/// [`chunk_contract`], and filesystem failures.
pub fn build_revision(root: &Path, store: &Store) -> io::Result<AssetRevision> {
let inventory = project::inspect_inventory(root)?;
let mut builder = Builder { store, entries: 0 };
Expand All @@ -39,20 +39,7 @@ pub fn build_revision(root: &Path, store: &Store) -> io::Result<AssetRevision> {
for app in &inventory.apps {
let mut assets = chunk_contract::AppAssets::default();
for (name, world) in &app.worlds {
let path = match world.format {
WorldFormat::Polar => root.join(&world.source),
WorldFormat::Anvil => root.join(format!(".chunk/build/worlds/{}/{name}.polar", app.id)),
};
if world.format == WorldFormat::Anvil && fs::symlink_metadata(&path).is_err() {
return Err(io::Error::new(
io::ErrorKind::NotFound,
format!(
"Anvil world {name} of app {} is not converted yet ({} is missing); run `chunk build`",
app.id,
path.display()
),
));
}
let path = root.join(&world.source);
let mut magic = [0; 4];
regular(&path)?.read_exact(&mut magic).ok();
if &magic != POLAR_MAGIC {
Expand Down
18 changes: 5 additions & 13 deletions crates/chunk-build/src/assets/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,18 @@ fn write(root: &Path, path: &str, bytes: &[u8]) {
fs::write(path, bytes).unwrap();
}

/// A project whose arena app has a Polar and an Anvil world and a pack, below a root scope with another pack.
/// A project whose arena app has two worlds and a pack, below a root scope with another pack.
fn project(root: &Path) {
write(root, "chunk.toml", b"");
write(root, "apps/scope.ts", b"export default defineScope({packs:{base:{source:'packs/base.zip'}}});");
write(
root,
"apps/arena/app.ts",
b"export default defineApp({id:'arena',worlds:{lobby:{source:'worlds/lobby.polar'},koth:{source:'worlds/koth'}},packs:{ui:{source:'packs/ui',required:true}}});",
b"export default defineApp({id:'arena',worlds:{lobby:{source:'worlds/lobby.polar'},koth:{source:'worlds/koth.polar'}},packs:{ui:{source:'packs/ui',required:true}}});",
);
write(root, "apps/arena/build.gradle.kts", b"");
write(root, "apps/arena/assets/worlds/lobby.polar", POLAR);
write(root, "apps/arena/assets/worlds/koth/level.dat", b"anvil");
write(root, "apps/arena/assets/worlds/koth.polar", b"Polr koth");
write(root, "apps/arena/assets/packs/ui/pack.mcmeta", b"{}");
write(root, "apps/arena/assets/packs/ui/assets/minecraft/lang/en_us.json", b"{}");
write(root, "apps/arena/assets/kits/default.json", b"kit");
Expand All @@ -37,12 +37,6 @@ fn revisions_hold_plain_files_worlds_and_reproducible_packs() {
let root = project_directory.path();
project(root);
let first = Store::new(root.join("dist/assets"));
let error = build_revision(root, &first).unwrap_err().to_string();
assert!(
error.contains("Anvil world koth of app arena is not converted") && error.contains("chunk build"),
"{error}"
);
write(root, ".chunk/build/worlds/arena/koth.polar", b"Polr converted");

let revision = build_revision(root, &first).unwrap();
assert_eq!(revision.shared.keys().collect::<Vec<_>>(), ["config/game.json"]);
Expand All @@ -63,7 +57,7 @@ fn revisions_hold_plain_files_worlds_and_reproducible_packs() {
let second = build_revision(root, &Store::new(root.join("other"))).unwrap();
assert_eq!(second.id(), revision.id());

write(root, ".chunk/build/worlds/arena/koth.polar", b"not polar");
write(root, "apps/arena/assets/worlds/koth.polar", b"not polar");
let error = build_revision(root, &first).unwrap_err().to_string();
assert!(error.contains("not a Polar world"), "{error}");
}
Expand All @@ -73,15 +67,14 @@ fn materialized_directories_link_read_only_blobs_and_are_reused() {
let project_directory = tempfile::tempdir().unwrap();
let root = project_directory.path();
project(root);
write(root, ".chunk/build/worlds/arena/koth.polar", b"Polr converted");
let store = Store::new(root.join("store"));
let revision = build_revision(root, &store).unwrap();

let directory = materialize(&store, &revision, "arena").unwrap();
assert_eq!(directory, store.root().join("apps").join(revision.id()).join("arena"));
assert_eq!(fs::read(directory.join("revision.json")).unwrap(), revision.encode());
assert_eq!(fs::read(directory.join("worlds/lobby.polar")).unwrap(), POLAR);
assert_eq!(fs::read(directory.join("worlds/koth.polar")).unwrap(), b"Polr converted");
assert_eq!(fs::read(directory.join("worlds/koth.polar")).unwrap(), b"Polr koth");
assert_eq!(fs::read(directory.join("app/kits/default.json")).unwrap(), b"kit");
assert_eq!(fs::read(directory.join("shared/config/game.json")).unwrap(), b"config");
assert!(!directory.join("packs").exists() && !directory.join("app/worlds").exists());
Expand All @@ -102,7 +95,6 @@ fn incomplete_materialized_directories_are_rebuilt() {
let project_directory = tempfile::tempdir().unwrap();
let root = project_directory.path();
project(root);
write(root, ".chunk/build/worlds/arena/koth.polar", b"Polr converted");
let store = Store::new(root.join("store"));
let revision = build_revision(root, &store).unwrap();
let directory = materialize(&store, &revision, "arena").unwrap();
Expand Down
2 changes: 1 addition & 1 deletion crates/chunk-build/src/project.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use crate::{MachineProfile, valid_id};
pub(crate) mod assets;
pub(crate) mod authoring;

pub use assets::{ChunkRange, Pack, World, WorldFormat};
pub use assets::{Pack, World};

/// Apps, domain scopes and authored modules found in one pass over the project tree.
#[derive(Default)]
Expand Down
88 changes: 11 additions & 77 deletions crates/chunk-build/src/project/assets.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use std::{
};

use chunk_contract::{AssetContract, PackDeclaration};
use oxc_ast::ast::{Expression, UnaryOperator};
use oxc_ast::ast::Expression;
use serde::Serialize;

use super::{
Expand All @@ -19,26 +19,8 @@ use crate::valid_id;

#[derive(Debug, Serialize)]
pub struct World {
/// Project-relative path with forward slashes.
/// Project-relative path with forward slashes to a `.polar` file.
pub source: String,
pub format: WorldFormat,
/// The inclusive chunk range an Anvil world is cropped to.
#[serde(skip_serializing_if = "Option::is_none")]
pub chunks: Option<ChunkRange>,
}

#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum WorldFormat {
/// A save directory with `level.dat`, converted to Polar by the Gradle build.
Anvil,
Polar,
}

#[derive(Debug, Serialize)]
pub struct ChunkRange {
pub from: [i32; 2],
pub to: [i32; 2],
}

#[derive(Debug)]
Expand All @@ -51,7 +33,7 @@ pub struct Pack {
}

/// The `assets/` directory a declaration's sources are relative to, and its project-relative path. Sources must
/// exist unless `require_sources` is false, which classifies a missing world by its `chunks` and extension.
/// exist unless `require_sources` is false.
pub(super) struct Root<'a> {
pub directory: &'a Path,
pub prefix: &'a str,
Expand All @@ -65,32 +47,17 @@ pub(super) fn worlds(file: &Path, expression: &Expression<'_>, root: &Root<'_>)
return Err(invalid(file, "world names must be ASCII identifiers"));
}
let fields = object_fields(file, value)?;
if fields.keys().any(|key| !["source", "chunks"].contains(key)) {
if fields.keys().any(|key| *key != "source") {
return Err(invalid(file, format!("unsupported option of world {name}")));
}
let (source, mut found) = source(file, fields.get("source").copied(), root)?;
if found.is_none() && !root.require_sources {
let path = root.directory.join(source.strip_prefix(root.prefix).unwrap_or(&source).trim_start_matches('/'));
found = fs::symlink_metadata(&path).ok().map(|metadata| (path, metadata));
}
let chunks = fields.get("chunks").map(|value| chunk_range(file, value)).transpose()?;
let format = match found {
Some((_, metadata)) if metadata.is_file() && extension(&source, "polar") => WorldFormat::Polar,
Some((path, metadata))
if metadata.is_dir() && fs::symlink_metadata(path.join("level.dat")).is_ok_and(|l| l.is_file()) =>
{
WorldFormat::Anvil
}
Some(_) => {
return Err(invalid(file, format!("world {name} requires a .polar file or an Anvil save directory")));
}
None if chunks.is_none() && extension(&source, "polar") => WorldFormat::Polar,
None => WorldFormat::Anvil,
};
if chunks.is_some() && format == WorldFormat::Polar {
return Err(invalid(file, format!("world {name} can only crop Anvil saves to chunks")));
let (source, found) = source(file, fields.get("source").copied(), root)?;
if !extension(&source, "polar") || found.is_some_and(|(_, metadata)| !metadata.is_file()) {
return Err(invalid(
file,
format!("world {name} must be a .polar file; Chunk no longer converts Anvil saves"),
));
}
worlds.insert(name.into(), World { source, format, chunks });
worlds.insert(name.into(), World { source });
}
Ok(worlds)
}
Expand Down Expand Up @@ -161,39 +128,6 @@ fn extension(source: &str, expected: &str) -> bool {
Path::new(source).extension().is_some_and(|extension| extension.eq_ignore_ascii_case(expected))
}

fn chunk_range(file: &Path, expression: &Expression<'_>) -> io::Result<ChunkRange> {
let error = || invalid(file, "chunks requires literal { from: [x, z], to: [x, z] } with from <= to");
let fields = object_fields(file, expression)?;
if fields.len() != 2 {
return Err(error());
}
let corner = |key| -> io::Result<[i32; 2]> {
let Some(Expression::ArrayExpression(array)) = fields.get(key) else { return Err(error()) };
let values = array
.elements
.iter()
.map(|element| element.as_expression().and_then(integer))
.collect::<Option<Vec<_>>>()
.ok_or_else(error)?;
values.try_into().map_err(|_| error())
};
let (from, to) = (corner("from")?, corner("to")?);
if from[0] > to[0] || from[1] > to[1] {
return Err(error());
}
Ok(ChunkRange { from, to })
}

fn integer(expression: &Expression<'_>) -> Option<i32> {
match expression {
Expression::NumericLiteral(number) => number.raw.as_ref()?.parse().ok(),
Expression::UnaryExpression(unary) if unary.operator == UnaryOperator::UnaryNegation => {
integer(&unary.argument)?.checked_neg()
}
_ => None,
}
}

/// Rejects pack names declared twice anywhere in the project.
pub(super) fn check_unique_packs(root: &Path, inventory: &Inventory) -> io::Result<()> {
let mut names = BTreeSet::new();
Expand Down
Loading
Loading