0.11.0: dist-apk takes Kotlin, libraries, archives and a Maven graph; dist-apple takes Info.plist entries and an iOS device

dist-apk
- options::kotlin_sources: kotlinc from xim:kotlin (feature dist-apk-kotlin)
  compiles the Kotlin with the Java as reference sources, before javac; the
  Kotlin stdlib is dexed.
- R classes: aapt2 link --java, with --extra-packages for every library.
- options::libraries (Android libraries from source), options::aars and
  options::jars, and options::maven through a lock file resolved by
  xim:coursier (feature dist-apk-maven). MCPP_DIST_APK_MAVEN=update and
  =fetch are the only packs that reach the network; an ordinary pack checks
  each cached artifact against the lock's sha256.
- A stated subset of the manifest merger: tools:node remove/replace,
  tools:replace, ${applicationId}, a conflicting element refused by name.
- options::sign = false writes the aligned package unsigned.

dist-apple
- options::info_plist: a project's entries join the Info.plist; a key the
  member derives is refused by name, and one it only defaults is replaced.
- options::provisioning_profile, on the iOS device row: embedded as
  embedded.mobileprovision, its entitlements sign the bundle, and its
  application identifier must cover the bundle's.
- The device row's runner named app is devicectl-run, from
  xim:apple-device-tools.

mcpp::plugins::xml is the XML reader and writer the two members share.
With no new option set, both members plan what 0.10.1 planned.

CI bridges the three xim recipes openxlings/xim-pkgindex#844 adds until the
published index carries them, and packs the Android fixtures on macOS too.

Author: Sunrisepeak

.github/scripts/bridge-xim-recipes.py

@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+"""The xim recipes 0.11.0's features declare, until the published index has them.
+
+TEMPORARY. `dist-apk-kotlin` declares `xim:kotlin` 2.4.20, `dist-apk-maven`
+declares `xim:coursier` 2.1.24, and `dist-apple` declares
+`xim:apple-device-tools` 0.1.0 on the iOS device row. openxlings/xim-pkgindex#844
+adds the three recipes. Until the index mcpp syncs carries them, this script
+writes the reviewed recipe files into the synced copy and adds the entries the
+index cache resolves `xim:<name>` through -- what `mcpp index update` would
+have written.
+
+A recipe the synced index already carries at the version declared here is left
+as the index has it, and the script says so. A file this script wrote begins
+with a marker line, so a second run in one job does not mistake it for the
+index's own. When a run prints `published` for all three, the recipes are in the
+index, and this script and the steps that run it are deleted.
+
+The files come from the pull request's head commit, named by its hash, so a CI
+run reads the bytes that were reviewed and not whatever a branch holds later.
+
+Run after `mcpp index update`, with MCPP_HOME set.
+"""
+import json
+import os
+import re
+import sys
+import urllib.request
+
+COMMIT = "8286a17724a4a4a399b23a3e109be457c052e23b"
+BASE = f"https://raw.githubusercontent.com/Sunrisepeak/xim-pkgindex/{COMMIT}/pkgs/"
+MARKER = f"-- bridged by mcpp-plugins CI from xim-pkgindex {COMMIT}\n"
+RECIPES = {  # index path: the version this collection declares
+    "k/kotlin.lua": "2.4.20",
+    "c/coursier.lua": "2.1.24",
+    "a/apple-device-tools.lua": "0.1.0",
+}
+
+
+def main() -> int:
+    home = os.environ.get("MCPP_HOME")
+    if not home:
+        print("::error::MCPP_HOME is not set; this script edits the index under it")
+        return 1
+    root = os.path.join(home, "registry", "data", "xim-pkgindex")
+    cache_path = os.path.join(root, ".xlings-index-cache.json")
+    if not os.path.isfile(cache_path):
+        print(f"::error::{cache_path} does not exist; run `mcpp index update` first")
+        return 1
+    with open(cache_path, encoding="utf-8") as f:
+        cache = json.load(f)
+    entries = cache["entries"]
+
+    changed = False
+    for rel, version in RECIPES.items():
+        dest = os.path.join(root, "pkgs", rel)
+        name = os.path.basename(rel)[:-len(".lua")]
+        key = "xim:" + name
+        if key in entries and os.path.isfile(dest):
+            with open(dest, encoding="utf-8") as f:
+                present = f.read()
+            if present.startswith(MARKER):
+                print(f"bridged earlier in this job: {key} {version}")
+                continue
+            if re.search(r'\["%s"\]' % re.escape(version), present):
+                print(f"published: the index carries {key} {version}; this bridge is not needed for it")
+                continue
+        with urllib.request.urlopen(BASE + rel, timeout=60) as r:
+            text = r.read().decode("utf-8")
+        if not re.search(r'\["%s"\]' % re.escape(version), text):
+            print(f"::error::{BASE + rel} has no version {version}")
+            return 1
+        declared = re.search(r'^\s*name\s*=\s*"([^"]+)"', text, re.M).group(1)
+        if declared != name:
+            print(f"::error::{rel} declares the name {declared}")
+            return 1
+        description = re.search(r'^\s*description\s*=\s*"([^"]*)"', text, re.M).group(1)
+        os.makedirs(os.path.dirname(dest), exist_ok=True)
+        with open(dest, "w", encoding="utf-8", newline="\n") as f:
+            f.write(MARKER + text)
+        entries[key] = {
+            "canonical_name": key, "description": description, "entry_key": key,
+            "identity": {"name": name, "namespace": "xim"}, "name": name,
+            "path": dest.replace("\\", "/"), "ref": "", "type": 0, "version": "",
+        }
+        changed = True
+        print(f"bridged: {key} {version} from xim-pkgindex {COMMIT[:12]}")
+
+    if changed:
+        with open(cache_path, "w", encoding="utf-8") as f:
+            json.dump(cache, f, indent=1)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

.github/workflows/ci.yml

@@ -1,7 +1,7 @@
 [package]
 name        = "plugins"
 namespace   = "mcpp"
-version     = "0.10.1"
+version     = "0.11.0"
 description = "Official mcpp build plugins: rule packages under mcpp.rules.*, build-time utilities under mcpp.tools.*, each member selected by a feature"
 license     = "Apache-2.0"
 authors     = ["mcpp-community"]
@@ -180,6 +180,20 @@ implies = ["surface"]
 sources = ["dist/apk.cppm"]
 implies = ["surface"]
 
+# `dist-apk` WITH THE KOTLIN COMPILER, AND WITH THE MAVEN RESOLVER (0.11.0).
+# Features of their own rather than payloads of `dist-apk`, for the reason
+# `dist-appimage` gives below about downloads: provisioning runs before the
+# build program learns what it will compile, so a payload `dist-apk` declared
+# would be installed by every Android consumer, and the Kotlin compiler is 90 MB
+# an application with no Kotlin never runs. A project with Kotlin sources names
+# `dist-apk-kotlin`; one with Maven coordinates names `dist-apk-maven`; either
+# implies the member itself.
+[features.dist-apk-kotlin]
+implies = ["dist-apk"]
+
+[features.dist-apk-maven]
+implies = ["dist-apk"]
+
 [features.tools-embed]
 sources = ["tools/embed.cppm"]
 implies           = ["surface"]
@@ -421,6 +435,13 @@ implies           = ["surface"]
 [target.'cfg(os = "macos")'.feature-xlings.dist-apple]
 "xim:macapp-run" = { version = "0.1.0", when = "run" }
 
+# THE iOS DEVICE ROW'S RUNNER (0.11.0), `devicectl-run`, which installs a signed
+# bundle on a connected device and launches it. The simulator row keeps the
+# runner its manifest names, so the axis excludes `env = "sim"`, and `when =
+# "run"` keeps a build or a pack from installing it.
+[target.'cfg(all(os = "ios", not(env = "sim")))'.feature-xlings.dist-apple]
+"xim:apple-device-tools" = { version = "0.1.0", when = "run" }
+
 # ── The environment `dist-apk` needs ───────────────────────────────────────
 #
 # THE GATE IS THE TARGET ENVIRONMENT, NOT AN ACCELERATOR. `env = "android"` is
@@ -468,6 +489,15 @@ implies           = ["surface"]
 "xim:android-debug-keystore"  = "1.0.0"
 "xim:bundletool"              = "1.18.3"
 
+# The compiler `options::kotlin_sources` needs, and the resolver `options::maven`
+# needs, each behind its own feature (see `[features.dist-apk-kotlin]`). The
+# Kotlin compiler's launcher runs the JDK pinned above, the same store key.
+[target.'cfg(env = "android")'.feature-xlings.dist-apk-kotlin]
+"xim:kotlin" = "2.4.20"
+
+[target.'cfg(env = "android")'.feature-xlings.dist-apk-maven]
+"xim:coursier" = "2.1.24"
+
 [targets.plugins]
 kind = "lib"
 

README.md

@@ -49,7 +49,7 @@ export namespace mcpp::plugins {
 //
 // One package, one version: the number lives in mcpp.toml, and the CI step
 // `the collection states its own version` compares the two.
-inline constexpr std::string_view version = "0.10.1";
+inline constexpr std::string_view version = "0.11.0";
 
 } // namespace mcpp::plugins
 
@@ -98,6 +98,171 @@ inline manifest read_manifest(std::string tree) {
 
 } // namespace mcpp::plugins::stage
 
+// mcpp::plugins::xml -- the XML two dist members read and write.
+//
+// SHARED BECAUSE TWO DIST MEMBERS READ IT. `dist-apk` merges library manifests
+// into an application's, and `dist-apple` adds a project's Info.plist entries
+// and reads the plist a provisioning profile carries. Neither needs more than
+// elements, attributes and text, and neither interprets an entity, so this is
+// that much and no more: no namespaces resolved, no DTD read, no validation.
+export namespace mcpp::plugins::xml {
+
+// A tree of elements and text. Comments, the XML declaration and a DOCTYPE are
+// read and dropped: the documents this member writes from a tree are generated
+// files, and a merged manifest carries no author to read a comment. Attribute
+// values are kept exactly as written, entities included, because nothing here
+// interprets them beyond comparing and substituting `${applicationId}`.
+struct node {
+    std::string                                      name;   // empty: a text node
+    std::vector<std::pair<std::string, std::string>> attrs;
+    std::vector<node>                            children;
+    std::string                                      text;   // a text node's characters
+};
+
+struct reader {
+    std::string_view s;
+    std::size_t      i = 0;
+    std::string      error;
+
+    static bool space(char c) { return c == ' ' || c == '\t' || c == '\n' || c == '\r'; }
+    bool starts(std::string_view t) const { return s.substr(i, t.size()) == t; }
+    void skip_space() { while (i < s.size() && space(s[i])) ++i; }
+    bool skip_past(std::string_view end) {
+        const auto p = s.find(end, i);
+        if (p == std::string_view::npos) { i = s.size(); return false; }
+        i = p + end.size();
+        return true;
+    }
+    std::string name() {
+        const std::size_t b = i;
+        while (i < s.size() && !space(s[i]) && s[i] != '>' && s[i] != '/' && s[i] != '=') ++i;
+        return std::string(s.substr(b, i - b));
+    }
+
+    bool element(node& out) {
+        ++i;  // '<'
+        out.name = name();
+        if (out.name.empty()) { error = "an element without a name"; return false; }
+        for (;;) {
+            skip_space();
+            if (i >= s.size()) { error = "<" + out.name + "> is not terminated"; return false; }
+            if (starts("/>")) { i += 2; return true; }
+            if (s[i] == '>') { ++i; break; }
+            std::string attr = name();
+            skip_space();
+            if (attr.empty() || i >= s.size() || s[i] != '=') {
+                error = "an attribute of <" + out.name + "> has no value";
+                return false;
+            }
+            ++i;
+            skip_space();
+            if (i >= s.size() || (s[i] != '"' && s[i] != '\'')) {
+                error = "attribute " + attr + " of <" + out.name + "> is not quoted";
+                return false;
+            }
+            const char quote = s[i++];
+            const auto end = s.find(quote, i);
+            if (end == std::string_view::npos) { error = "attribute " + attr + " is not terminated"; return false; }
+            out.attrs.emplace_back(std::move(attr), std::string(s.substr(i, end - i)));
+            i = end + 1;
+        }
+        for (;;) {
+            if (i >= s.size()) { error = "<" + out.name + "> is not closed"; return false; }
+            if (starts("</")) {
+                i += 2;
+                const std::string closing = name();
+                skip_space();
+                if (closing != out.name || i >= s.size() || s[i] != '>') {
+                    error = "</" + closing + "> closes <" + out.name + ">";
+                    return false;
+                }
+                ++i;
+                return true;
+            }
+            if (starts("<!--")) { if (!skip_past("-->")) { error = "a comment is not terminated"; return false; } continue; }
+            if (starts("<?"))   { if (!skip_past("?>"))  { error = "a processing instruction is not terminated"; return false; } continue; }
+            if (starts("<![CDATA[")) {
+                const std::size_t b = i;
+                if (!skip_past("]]>")) { error = "a CDATA section is not terminated"; return false; }
+                node t;
+                t.text = std::string(s.substr(b, i - b));
+                out.children.push_back(std::move(t));
+                continue;
+            }
+            if (s[i] == '<') {
+                node child;
+                if (!element(child)) return false;
+                out.children.push_back(std::move(child));
+                continue;
+            }
+            const std::size_t b = i;
+            auto end = s.find('<', i);
+            if (end == std::string_view::npos) end = s.size();
+            i = end;
+            bool blank = true;
+            for (std::size_t j = b; j < end; ++j) if (!space(s[j])) { blank = false; break; }
+            if (!blank) {
+                node t;
+                t.text = std::string(s.substr(b, end - b));
+                out.children.push_back(std::move(t));
+            }
+        }
+    }
+
+    bool document(node& root) {
+        for (;;) {
+            skip_space();
+            if (starts("<?"))        { skip_past("?>");  continue; }
+            if (starts("<!--"))      { skip_past("-->"); continue; }
+            if (starts("<!DOCTYPE")) { skip_past(">");   continue; }
+            break;
+        }
+        if (i >= s.size() || s[i] != '<') { error = "no root element"; return false; }
+        if (!element(root)) return false;
+        return true;
+    }
+};
+
+inline bool parse(std::string_view text, node& root, std::string& error) {
+    reader r{text};
+    if (!r.document(root)) { error = r.error; return false; }
+    return true;
+}
+
+inline std::string trim_copy(const std::string& s) {
+    std::size_t b = 0, e = s.size();
+    while (b < e && reader::space(s[b])) ++b;
+    while (e > b && reader::space(s[e - 1])) --e;
+    return s.substr(b, e - b);
+}
+
+inline void write(const node& n, std::string& out, int depth) {
+    const std::string pad(static_cast<std::size_t>(depth) * 4, ' ');
+    if (n.name.empty()) { out += pad + trim_copy(n.text) + "\n"; return; }
+    out += pad + "<" + n.name;
+    for (auto const& a : n.attrs) out += " " + a.first + "=\"" + a.second + "\"";
+    if (n.children.empty()) { out += "/>\n"; return; }
+    if (n.children.size() == 1 && n.children.front().name.empty()) {
+        out += ">" + n.children.front().text + "</" + n.name + ">\n";
+        return;
+    }
+    out += ">\n";
+    for (auto const& c : n.children) write(c, out, depth + 1);
+    out += pad + "</" + n.name + ">\n";
+}
+
+inline std::string attr_of(const node& n, std::string_view key) {
+    for (auto const& a : n.attrs) if (a.first == key) return a.second;
+    return {};
+}
+
+inline void set_attr(node& n, const std::string& key, const std::string& value) {
+    for (auto& a : n.attrs) if (a.first == key) { a.second = value; return; }
+    n.attrs.emplace_back(key, value);
+}
+
+} // namespace mcpp::plugins::xml
+
 // mcpp::plugins::names -- the derivations that turn a path into a C++ name.
 //
 // THESE ARE SHARED BECAUSE THEY WERE COPIED. `common_base_dir` and

dist/apk.cppm

@@ -44,6 +44,7 @@ plugins = { path = "../..", features = [
   "rules-spirv", "rules-sycl",
   "tools-embed", "tools-island",
   "dist-appimage", "dist-wix", "dist-apple", "dist-web", "dist-apk",
+  "dist-apk-kotlin", "dist-apk-maven",
 ], host-module = true }
 
 # NO `accel`, and that is the whole design: with none, every rule returns
@@ -61,6 +62,11 @@ plugins = { path = "../..", features = [
 # every member's module is COMPILED on every host, and a member left out is a
 # member whose macOS and Windows compile is never attempted -- which is exactly
 # the failure the two platform-specific members are here to catch.
+#
+# `dist-apk-kotlin` and `dist-apk-maven` add no source: each implies `dist-apk`
+# and declares one payload on the `cfg(env = "android")` axis, which a host
+# build never opens. They are listed because a consumer names them, which is
+# what the CI step that reads this list counts.
 
 [build]
 sources = ["src/*.cpp"]

dist/apple.cppm

@@ -0,0 +1,2 @@
+maven.lock
+.coursier-cache/

mcpp.toml

@@ -0,0 +1,10 @@
+// Java beside the Kotlin activity, reaching the application's R and both archives.
+package org.mcpp.apklibs;
+
+public final class JavaHelper {
+    private JavaHelper() {}
+
+    public static String describe() {
+        return "java:" + R.string.shared_name + ":" + org.mcpp.jar.JarThing.NAME + ":" + org.mcpp.aar.AarThing.name();
+    }
+}