diff --git a/.gitignore b/.gitignore index c3a0cb1fd..59f2c936e 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,8 @@ obj/ !nostrand/references/*.dll !magic-unity/Runtime/magic/*.dll !unity-examples/magic-unity-coexist/Assets/Plugins/Consumer/*.dll +!unity-examples/magic-unity-smoke/csharp-lib/src_classes/*.dll +!unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/*.dll !magic-unity/Runtime/clojure-clr/*.dll # macOS diff --git a/CHANGELOG.md b/CHANGELOG.md index 5b22f2c20..ea140b862 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,42 @@ # Changelog +## v0.13.0 - 2026-09-09 + +**`nos build` copies a library's C# assemblies into the build output**, so shipping a compiled assembly to Unity takes no hand-written `File/Copy`, and Unity imports the plugin once instead of on every build. **Saving a `.clj`, `.cljc` or `.cljr` re-evaluates it in the ClojureCLR Editor**, on the one save rather than the third. + +`nos build` also compiles with `*unchecked-math*` false, Clojure's default, so arithmetic and a narrowing cast throw on overflow instead of wrapping silently. Fifteen fixes across the compiler, runtime, stdlib and Mage align behaviour with JVM Clojure and ClojureCLR. The Editor runs ClojureCLR 1.11.0-flybot5. + +### Compiler +- Calling something that is not a function throws a catchable `InvalidCastException`, the way JVM Clojure throws `ClassCastException`, instead of `InvalidProgramException: Invalid IL code`. The compiler boxes a value-typed callee before casting it to `IFn`, so the method it lands in verifies; `(1 2)` and `(let [x (int 1)] (x 2))` each produced a method the JIT refused to load - [#171](https://github.com/flybot-sg/magic/issues/171). +- `deftype`, `reify` and `proxy` bind a method to the slot it overrides rather than to one declaration of it, so a type can implement an interface that redeclares an inherited member. Writing `count` on `clojure.lang.IPersistentMap` used to fail with `No match binding method`, and naming the interface to get past it left the sibling declarations throwing `NotImplementedException` at run time. A return type hint on the method name picks between overloads that differ only in return type, as it does on ClojureCLR, and the compiler names the choices when the hint is missing - [#146](https://github.com/flybot-sg/magic/issues/146). +- A narrowing cast on a type-hinted primitive throws when the value does not fit instead of discarding the high bits, so `(int 4294967296)` on a `^long` throws `ArgumentException` rather than returning `0`, as on ClojureCLR and JVM Clojure - [#148](https://github.com/flybot-sg/magic/issues/148). +- A cast on a numeric literal behaves like the same cast at runtime: `(int 1.5)` returns 1 instead of 2, and an out-of-range literal like `(int 4294967296)` throws a catchable `ArgumentException` at runtime instead of aborting compilation with a bare `OverflowException` - [#153](https://github.com/flybot-sg/magic/issues/153). + +### Runtime +- Casting a boxed `UInt64` converts instead of throwing `InvalidCastException`, so `(int (identity (ulong 1)))` returns 1 - [#151](https://github.com/flybot-sg/magic/issues/151). + +### Stdlib +- `spit` and `writer` truncate the file they overwrite, and `:append` appends, matching the JVM. A write used to open at position 0 without truncating, so shorter content produced a mix of new and old bytes, and `:append` was silently dropped - [#155](https://github.com/flybot-sg/magic/issues/155). +- `#object[...]` prints the qualified type name, so `(pr-str (System.Text.StringBuilder.))` names `System.Text.StringBuilder` instead of `StringBuilder`. `print-tagged-object` wrote `.Name`, which drops the namespace and cannot identify a type, while `print-method` on the type object already wrote `.FullName` - [#142](https://github.com/flybot-sg/magic/issues/142). +- `pr-str` of an exception writes the `:message` value, so `#error` output carries the message instead of a bare `:message` key with nothing after it. The `:via` map was left with an odd number of forms - [#158](https://github.com/flybot-sg/magic/issues/158). +- `defn` records the qualified type name in an arglist `:tag`, so a hint like `^Regex` is stored as `System.Text.RegularExpressions.Regex` and resolves from any namespace instead of only where the import is in scope - [#162](https://github.com/flybot-sg/magic/issues/162). +- `sort` and `sort-by` carry the collection's metadata through to the sorted seq - [#163](https://github.com/flybot-sg/magic/issues/163). +- A namespace map prints its keys in the map's own order, so `#:a{...}` no longer reorders them once the map outgrows an array-map - [#165](https://github.com/flybot-sg/magic/issues/165). +- `clojure.pprint/pprint` writes collection metadata when `*print-meta*` is true, so a pretty-printed value carries its metadata like `pr` does - [#166](https://github.com/flybot-sg/magic/issues/166). +- `clojure.repl/doc` prints a special form's docstring once instead of repeating it after the "Please see" line - [#167](https://github.com/flybot-sg/magic/issues/167). +- `clojure.string/split` drops trailing empty strings, and a negative limit returns every part. `(split "a b " #" ")` returned `["a" "b" ""]`. `split-lines` gained a final `""` on a trailing newline, and the negative limit threw. A pattern that matches nothing at the start no longer adds a leading `""`, so `(split "abc" #"")` returns `["a" "b" "c"]` - [#174](https://github.com/flybot-sg/magic/issues/174). + +### Mage +- `il/type`'s short arities work, so a caller can write `(il/type "Name" body)` instead of spelling out attributes, interfaces, supertype, generic parameters and custom attributes every time. Every arity below the 7-arity threw `ArityException` - [#143](https://github.com/flybot-sg/magic/issues/143). + +### Nostrand +- `nos build` compiles with `*unchecked-math*` false, Clojure's default, so arithmetic and narrowing casts keep their overflow checks instead of wrapping silently. A namespace that wants wrapping sets the flag itself - [#149](https://github.com/flybot-sg/magic/issues/149). +- `nos build` copies the C# assemblies a library ships into the output dir, so a consumer no longer writes its own `File/Copy` to get them there. Point `:csharp-out` at a second dir to keep them out of `:out`, which `:clean?` deletes on every build: Unity then imports the plugin once and its GUID holds, instead of a new one on every build - [#144](https://github.com/flybot-sg/magic/issues/144). + +### Unity +- The Editor loads ClojureCLR 1.11.0-flybot5, so both Editor runtimes agree on the stdlib fixes it carries. `spit` and `writer` truncate the file they overwrite, a printed double reads back equal, and `(long x)` on a boxed `UInt64` converts. `clojure.string/split` drops trailing empty strings, a stack frame names the code that ran, and a string, map or record hashes to the JVM's value - [clojure-1.11.0-flybot4](https://github.com/flybot-sg/clojure-clr/releases/tag/clojure-1.11.0-flybot4), [clojure-1.11.0-flybot5](https://github.com/flybot-sg/clojure-clr/releases/tag/clojure-1.11.0-flybot5). +- The package ships `Magic.Unity.ClojureReloader`, so saving a `.clj`, `.cljc` or `.cljr` re-evaluates it in the ClojureCLR Editor on the one save. A consumer's own `FileSystemWatcher` hook fires two or three events per save, so a reload took several saves before - [#157](https://github.com/flybot-sg/magic/issues/157). + ## v0.12.1 - 2026-08-20 The Editor runs **ClojureCLR 1.11.0-flybot3**, so `sort` and `compare` order values there the way MAGIC does. `nos` also reads a submodule's `deps-clr.edn`, so a library can leave `deps.edn` to the JVM. diff --git a/README.md b/README.md index 03c0debc8..ff80492fd 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ Using MAGIC on your own library: - [Writing cross-platform Clojure](./docs/writing-cross-platform-clojure.md): `.cljc` source patterns for code that runs on both the JVM and the CLR. - [Declaring CLR dependencies](./docs/clr-dependency-files.md): `deps-clr.edn` vs a `:clr` alias, when the CLR needs different deps than the JVM. - [The `nos` CLI](./docs/nos-cli.md): how a task is found, `nos build` and `nos test`, and the `magic.edn` surface. -- [Loading precompiled native assemblies](./docs/native-assemblies.md): loading a committed C# DLL on the CLR, where `:import` alone does not. +- [A library's C# assembly](./docs/native-assemblies.md): getting a committed C# DLL into the process, where `:import` alone does not, and how `nos build` ships it. - [Unity integration](./docs/unity-integration.md): compile Clojure to plugin DLLs and load them in a Unity project. Working on MAGIC itself: diff --git a/bb.edn b/bb.edn index 8dd40e161..ce8482ca4 100644 --- a/bb.edn +++ b/bb.edn @@ -91,6 +91,7 @@ [magic.unity :as unity]) :depends [regen-callsites refresh-stdlib sync-upm-version] :task (do (unity/check-constraints!) ; run after refresh-stdlib has written the metas + (drift/check-copies!) (drift/check!))} ;; ============================================================ diff --git a/bb/magic/coexist.clj b/bb/magic/coexist.clj index 0700d2ade..2fe71483d 100644 --- a/bb/magic/coexist.clj +++ b/bb/magic/coexist.clj @@ -34,7 +34,7 @@ (defn- field-mismatches [expected fields] (for [[k v] (sort expected) :when (not= v (get fields k))] - (str k "=" (pr-str (get fields k)) " (expected " (pr-str v) ")"))) + (str (name k) "=" (pr-str (get fields k)) " (expected " (pr-str v) ")"))) (defn- clj-dlls [dir] (mapcat #(fs/glob dir (str "*" %)) unity/clj-extensions)) @@ -43,25 +43,32 @@ (+ (count (clj-dlls (unity/runtime-dir :magic))) (count (clj-dlls consumer-dir)))) +(def ^:private csharp-fields + "The C# assembly carries no define constraint, so both states report it alike." + {:csharp-in-domain "true" + :csharp-editor-refs "1"}) + (def ^:private states "The two valid Editor states, keyed by the runtime the Editor loads, each with the probe fields that must hold." (delay (let [n (str (shipped-clj-count))] - {"clojure-clr" {:symbol? false - :probe {"symbol" "unset" - "preloaded-clj" "0" - "core-clj-loadable" "false" - "clojure-versions" "[1.11.0.0]" - "editor-clj-refs" "0" - "player-clj-refs" n}} - "magic" {:symbol? true - :probe {"symbol" "set" - "preloaded-clj" n - "core-clj-loadable" "true" - "clojure-versions" "[1.0.0.0]" - "editor-clj-refs" n - "player-clj-refs" n}}}))) + {:clojure-clr {:symbol? false + :probe (merge csharp-fields + {:symbol "unset" + :preloaded-clj "0" + :core-clj-loadable "false" + :clojure-versions "[1.11.0.0]" + :editor-clj-refs "0" + :player-clj-refs n})} + :magic {:symbol? true + :probe (merge csharp-fields + {:symbol "set" + :preloaded-clj n + :core-clj-loadable "true" + :clojure-versions "[1.0.0.0]" + :editor-clj-refs n + :player-clj-refs n})}}))) (defn- pack-tarball! "Pack pkg into a UPM tarball at tgz; exclude paths are relative to pkg." @@ -178,13 +185,13 @@ (if (seq found) [:fail (str "error lines in " (count found) " log(s): " (pr-str found))] [:pass "no unexpected error lines in any log"])] - (report! (array-map :check "logs" :status status :message message)))) + (report! (array-map :check :logs :status status :message message)))) (defn- marker "The first line carrying tag, and its key=value pairs." [lines tag] (let [line (first (filter #(str/includes? % tag) lines))] - [line (into {} (map (fn [[_ k v]] [k v]) + [line (into {} (map (fn [[_ k v]] [(keyword k) v]) (re-seq #"(\S+)=(\S+)" (or line ""))))])) (defn- parse-log [log-text] @@ -210,9 +217,9 @@ (nil? probe) [:inconclusive "no [CoexistenceProbe] line; did the package resolve?"] (seq problems) - [:fail (str "the " state " Editor state is wrong: " (str/join ", " problems))] + [:fail (str "the " (name state) " Editor state is wrong: " (str/join ", " problems))] :else - [:pass (str "the " state " Editor state holds, silently")]))) + [:pass (str "the " (name state) " Editor state holds, silently")]))) ;;; The Project Settings > MAGIC toggle @@ -234,13 +241,13 @@ [group (str/trim defines)]))) (defn- toggle-verdict [fields block-before block-after] - (let [seed (get fields "seed") + (let [seed (:seed fields) groups-before (defines-by-group block-before) groups-after (defines-by-group block-after) - problems (concat (field-mismatches {"after-set" (str seed ";" unity/magic-symbol) - "after-clear" seed - "enabled-after-set" "true" - "enabled-after-clear" "false"} + problems (concat (field-mismatches {:after-set (str seed ";" unity/magic-symbol) + :after-clear seed + :enabled-after-set "true" + :enabled-after-clear "false"} fields) (when-not (= groups-before groups-after) [(str "it changed some group's defines: " @@ -309,7 +316,7 @@ (str "Reconcile brought " brought " assembl" (if (= 1 brought) "y" "ies") " but the baseline left " (count before) " unconstrained, so something else constrained the rest")) - leaked (field-mismatches {"preloaded-clj" "0" "editor-clj-refs" "0"} fields) + leaked (field-mismatches {:preloaded-clj "0" :editor-clj-refs "0"} fields) ;; Two mechanisms each keep this at 1: Reconcile pre-constrains the ;; metas, and the import callback batches its report behind a delayCall. ;; A regression in one is invisible until the other goes too. @@ -329,7 +336,7 @@ [:inconclusive (str "the upgrade launch logged no \"" broken-line "\" line, so nothing " "exercises the broken-load path whose convergence this check asserts; " "the fixture needs a typed-invoke .clj.dll")] - (nil? (get fields "preloaded-clj")) + (nil? (:preloaded-clj fields)) [:inconclusive "no [CoexistenceProbe] line after the upgrade; did the package resolve?"] ;; A refusal also trips not-constrained; its warning is the better message. (:refused reconcile) @@ -395,7 +402,7 @@ :converged (broken-count probe-text)} [line fields] (marker (str/split-lines probe-text) "[CoexistenceProbe]") [status message] (upgrade-verdict before after fields reconcile broken)] - (report! (array-map :check "upgrade" :status status :message message + (report! (array-map :check :upgrade :status status :message message :before before :after after :reconcile (:brought reconcile) :broken broken :probe line)))))) @@ -419,22 +426,22 @@ (spit project-settings (str/replace-first (settings-content) define-symbols-re (str/re-quote-replacement before)))) - (report! (array-map :check "toggle" :status status :message message :probe line)))) + (report! (array-map :check :toggle :status status :message message :probe line)))) (defn- run-state! "Drive one Editor state end to end. The probe needs a second launch because the first one is what compiles the project against the new define set." [state] (let [{:keys [symbol?]} (get @states state) - editor-log (log-path (str state ".editor"))] + editor-log (log-path (str (name state) ".editor"))] (println) - (println (str "=== " state " Editor state: MAGIC_RUNTIME_IN_EDITOR " + (println (str "=== " (name state) " Editor state: MAGIC_RUNTIME_IN_EDITOR " (if symbol? "set" "unset"))) (write-define-symbol! symbol?) (install!) (reset-consumer-metas!) (println "Run 1/2: cold import (slow)...") - (run-editor! (log-path (str state ".import"))) + (run-editor! (log-path (str (name state) ".import"))) (println "Run 2/2: domain reload (narration + probe)...") (run-editor! editor-log "-executeMethod" "CoexistenceProbe.Run") (let [{:keys [narration dedup broken probe fields] :as result} (parse-log (slurp editor-log)) @@ -443,41 +450,46 @@ :narration narration :dedup dedup :broken broken :probe probe :fields fields))))) +(defn- report-outcome! + "Print one line per check and fail the task on anything that is not a pass." + [results checks] + (let [players (distinct (keep #(get-in % [:fields :player-clj-refs]) results)) + moved? (and (next results) (not= 1 (count players)))] + (println) + (doseq [{:keys [state check status message]} checks] + (println (format "%-11s %-14s %s" (name (or state check)) status message))) + ;; Each state's expectation already pins its own player-clj-refs count; + ;; this catches both drifting together. + (when (next results) + (println (if moved? + (str "player-clj-refs MOVED between states: " (pr-str players)) + (str "player-clj-refs identical across states: " (first players))))) + (when (or moved? (some #(not= :pass (:status %)) checks)) + (log/fail! "coexist-noise failed" "" + "An Editor state, the upgrade path, the runtime toggle, or the logs" + "themselves did not hold. The logs are under" + (str " " (coexist-path "Logs") "/"))))) + (defn coexist-noise! "Regression-check the Editor runtime states on unity-examples/magic-unity-coexist. - state is \"clojure-clr\", \"magic\", or nil for both, then the upgrade path and the - Editor Runtime toggle. + state-arg is the command-line \"clojure-clr\", \"magic\", or nil for both, then + the upgrade path and the Editor Runtime toggle. The two assert opposite things: `clojure-clr` (symbol unset, where every install starts) must keep the MAGIC runtime out of the Editor, `magic` must boot it. Both must be silent, and player references must not move between them." - [state] + [state-arg] (when-not (fs/exists? unity/unity-app) (log/fail! (str "Unity " unity/unity-version " not found") (str " " unity/unity-app) " Override the path with MAGIC_UNITY_APP.")) - (when (and state (not (contains? @states state))) - (log/fail! (str "unknown state: " state) - (str " valid states: " (str/join " | " (keys @states))))) - (sweep-logs!) - (let [results (mapv run-state! (if state [state] ["magic" "clojure-clr"])) - ;; Order matters: check-upgrade! ends with the real package resolved, - ;; which check-toggle! reuses instead of repacking, and check-logs! is - ;; last because it reads what every launch before it wrote. - checks (into results [(check-upgrade!) (check-toggle!) (check-logs!)]) - players (distinct (keep #(get-in % [:fields "player-clj-refs"]) results))] - (println) - (doseq [{:keys [state check status message]} checks] - (println (format "%-11s %-14s %s" (or state check) status message))) - ;; Each state's expectation already pins its own player-clj-refs count; - ;; this catches both drifting together. - (when (> (count results) 1) - (println (if (= 1 (count players)) - (str "player-clj-refs identical across states: " (first players)) - (str "player-clj-refs MOVED between states: " (pr-str players))))) - (when (or (some #(not= :pass (:status %)) checks) - (and (> (count results) 1) (not= 1 (count players)))) - (log/fail! "coexist-noise failed" "" - "An Editor state, the upgrade path, the runtime toggle, or the logs" - "themselves did not hold. The logs are under" - (str " " (coexist-path "Logs") "/"))))) + (let [state (some-> state-arg keyword)] + (when (and state (not (contains? @states state))) + (log/fail! (str "unknown state: " state-arg) + (str " valid states: " (str/join " | " (map name (keys @states)))))) + (sweep-logs!) + (let [results (mapv run-state! (if state [state] [:magic :clojure-clr]))] + ;; Order matters: check-upgrade! ends with the real package resolved, which + ;; check-toggle! reuses instead of repacking, and check-logs! is last + ;; because it reads what every launch before it wrote. + (report-outcome! results (into results [(check-upgrade!) (check-toggle!) (check-logs!)]))))) diff --git a/bb/magic/drift.clj b/bb/magic/drift.clj index a745ab4ac..9f55425e3 100644 --- a/bb/magic/drift.clj +++ b/bb/magic/drift.clj @@ -4,7 +4,9 @@ bootstrap. check-drift fails when a checked path differs from HEAD, meaning a source was edited without refreshing its DLL. This namespace also stamps the DLL mtimes from that manifest: a source that still hashes gets its DLL - stamped newer, so the loader takes the committed bytes over recompiling." + stamped newer, so the loader takes the committed bytes over recompiling. + It also verifies the committed copies of one file that the repo keeps in + several trees." (:require [babashka.fs :as fs] [babashka.tasks :refer [shell]] [clojure.edn] @@ -95,6 +97,29 @@ "}\n")) (println "recorded" manifest-path "-" (count entries) "sources"))) +(def ^:private committed-copies + "Files the repo commits in several trees, which must stay byte-identical. + Each Unity project needs the assembly inside its own Assets/, and the smoke + project also commits it as csharp-lib source and as :csharp-out output." + {"smoke_csharp.dll" + ["unity-examples/magic-unity-smoke/csharp-lib/src_classes/smoke_csharp.dll" + "unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/smoke_csharp.dll" + "unity-examples/magic-unity-coexist/Assets/Plugins/Consumer/smoke_csharp.dll"]}) + +(defn- copy-divergences [] + (for [[file-name paths] committed-copies + :when (next (distinct (map sha256 paths)))] + (str " " file-name "\n" + (str/join "\n" (for [p paths] (str " " (subs (sha256 p) 0 12) " " p)))))) + +(defn check-copies! + "Fail if the copies of one committed file have drifted apart." + [] + (when-let [diverged (seq (copy-divergences))] + (apply log/fail! "committed copies diverged" + (concat ["" "Rebuild wrote one copy and not the others:" ""] diverged))) + (println "committed copies OK -" (count committed-copies) "file(s)")) + (defn check! "After the regen tasks have run, fail if any checked path differs from HEAD. Committed DLLs are byte-diffed, except magic-unity's MAGIC Clojure.dll and diff --git a/bb/magic/unity.clj b/bb/magic/unity.clj index 98bfcde9b..e2a281048 100644 --- a/bb/magic/unity.clj +++ b/bb/magic/unity.clj @@ -83,7 +83,9 @@ {(str default-pkg "/Editor/PlayerCljAssemblies.cs") {:form #"Extensions = \{([^}]*)\}" :dll identity} "magic-compiler/src/magic/api.clj" - {:form #"source-extensions \[([^\]]*)\]" :dll #(str % ".dll")}}) + {:form #"source-extensions \[([^\]]*)\]" :dll #(str % ".dll")} + "nostrand/nostrand/tasks.clj" + {:form #"clj-assembly-suffixes[^\[]*\[([^\]]*)\]" :dll identity}}) (defn- extension-mismatches [] (for [[path {:keys [form dll]}] extension-sources diff --git a/clojure-runtime/Clojure/Lib/RT.cs b/clojure-runtime/Clojure/Lib/RT.cs index ece4967b3..4107a112e 100644 --- a/clojure-runtime/Clojure/Lib/RT.cs +++ b/clojure-runtime/Clojure/Lib/RT.cs @@ -1704,6 +1704,625 @@ public static decimal decimalCast(object x) #endregion + #region casting misc numeric types from primitives + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(char x) + { + if (x > (ulong)Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(byte x) + { + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(sbyte x) + { + if (x < 0 || x > Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(short x) + { + if (x < 0 || x > Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(ushort x) + { + if (x > (ulong)Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(int x) + { + if (x < 0 || x > Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(uint x) + { + if (x > (ulong)Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(long x) + { + if (x < 0 || x > Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(ulong x) + { + if (x > (ulong)Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(float x) + { + if (x < Byte.MinValue || x > Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static byte byteCast(double x) + { + if (x < Byte.MinValue || x > Byte.MaxValue) + throw new ArgumentException("Value out of range for byte: " + x); + + return (byte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(char x) + { + if (x > (ulong)SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(byte x) + { + if (x > (ulong)SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(sbyte x) + { + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(short x) + { + if (x < SByte.MinValue || x > SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(ushort x) + { + if (x > (ulong)SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(int x) + { + if (x < SByte.MinValue || x > SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(uint x) + { + if (x > (ulong)SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(long x) + { + if (x < SByte.MinValue || x > SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(ulong x) + { + if (x > (ulong)SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(float x) + { + if (x < SByte.MinValue || x > SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static sbyte sbyteCast(double x) + { + if (x < SByte.MinValue || x > SByte.MaxValue) + throw new ArgumentException("Value out of range for sbyte: " + x); + + return (sbyte)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(char x) + { + if (x > (ulong)Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(byte x) + { + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(sbyte x) + { + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(short x) + { + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(ushort x) + { + if (x > (ulong)Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(int x) + { + if (x < Int16.MinValue || x > Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(uint x) + { + if (x > (ulong)Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(long x) + { + if (x < Int16.MinValue || x > Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(ulong x) + { + if (x > (ulong)Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(float x) + { + if (x < Int16.MinValue || x > Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static short shortCast(double x) + { + if (x < Int16.MinValue || x > Int16.MaxValue) + throw new ArgumentException("Value out of range for short: " + x); + + return (short)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(char x) + { + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(byte x) + { + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(sbyte x) + { + if (x < 0 || x > UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(short x) + { + if (x < 0 || x > UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(ushort x) + { + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(int x) + { + if (x < 0 || x > UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(uint x) + { + if (x > (ulong)UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(long x) + { + if (x < 0 || x > UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(ulong x) + { + if (x > (ulong)UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(float x) + { + if (x < UInt16.MinValue || x > UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ushort ushortCast(double x) + { + if (x < UInt16.MinValue || x > UInt16.MaxValue) + throw new ArgumentException("Value out of range for ushort: " + x); + + return (ushort)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(char x) + { + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(byte x) + { + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(sbyte x) + { + if (x < 0 || x > UInt32.MaxValue) + throw new ArgumentException("Value out of range for uint: " + x); + + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(short x) + { + if (x < 0 || x > UInt32.MaxValue) + throw new ArgumentException("Value out of range for uint: " + x); + + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(ushort x) + { + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(int x) + { + if (x < 0 || x > UInt32.MaxValue) + throw new ArgumentException("Value out of range for uint: " + x); + + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(uint x) + { + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(long x) + { + if (x < 0 || x > UInt32.MaxValue) + throw new ArgumentException("Value out of range for uint: " + x); + + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(ulong x) + { + if (x > (ulong)UInt32.MaxValue) + throw new ArgumentException("Value out of range for uint: " + x); + + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(float x) + { + if (x < UInt32.MinValue || x > UInt32.MaxValue) + throw new ArgumentException("Value out of range for uint: " + x); + + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static uint uintCast(double x) + { + if (x < UInt32.MinValue || x > UInt32.MaxValue) + throw new ArgumentException("Value out of range for uint: " + x); + + return (uint)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(char x) + { + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(byte x) + { + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(sbyte x) + { + if (x < 0) + throw new ArgumentException("Value out of range for ulong: " + x); + + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(short x) + { + if (x < 0) + throw new ArgumentException("Value out of range for ulong: " + x); + + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(ushort x) + { + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(int x) + { + if (x < 0) + throw new ArgumentException("Value out of range for ulong: " + x); + + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(uint x) + { + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(long x) + { + if (x < 0) + throw new ArgumentException("Value out of range for ulong: " + x); + + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(ulong x) + { + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(float x) + { + if (x < UInt64.MinValue || x > UInt64.MaxValue) + throw new ArgumentException("Value out of range for ulong: " + x); + + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static ulong ulongCast(double x) + { + if (x < UInt64.MinValue || x > UInt64.MaxValue) + throw new ArgumentException("Value out of range for ulong: " + x); + + return (ulong)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(char x) + { + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(byte x) + { + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(sbyte x) + { + if (x < 0 || x > Char.MaxValue) + throw new ArgumentException("Value out of range for char: " + x); + + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(short x) + { + if (x < 0 || x > Char.MaxValue) + throw new ArgumentException("Value out of range for char: " + x); + + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(ushort x) + { + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(int x) + { + if (x < 0 || x > Char.MaxValue) + throw new ArgumentException("Value out of range for char: " + x); + + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(uint x) + { + if (x > (ulong)Char.MaxValue) + throw new ArgumentException("Value out of range for char: " + x); + + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(ulong x) + { + if (x > (ulong)Char.MaxValue) + throw new ArgumentException("Value out of range for char: " + x); + + return (char)x; + } + + [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] + public static char charCast(float x) + { + if (x < Char.MinValue || x > Char.MaxValue) + throw new ArgumentException("Value out of range for char: " + x); + + return (char)x; + } + + #endregion + #region int casting [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Naming", "CA1709:IdentifiersShouldBeCasedCorrectly")] @@ -1834,7 +2453,7 @@ public static long longCast(object x) ulong ux = (ulong)x; if (ux > long.MaxValue) throw new ArgumentException("Value out of range for long: " + x); - return (long)x; + return (long)ux; } if (x is byte || x is short || x is uint || x is sbyte || x is ushort) diff --git a/docs/architecture.md b/docs/architecture.md index 8edbe7fbb..39317880c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -234,13 +234,13 @@ The two backends in the diagram are Unity's scripting backend setting, chosen pe The runtime side is small. `Runtime/Magic.Unity.cs` is the `Magic.Unity.Clojure` API that C# scripts call, `Boot`, `Require` and `GetVar`, and `Runtime/magic/` holds the prebuilt runtime plus the whole stdlib. In Play mode no build callback fires, and the assemblies load and run as they are. -The editor side, nine files, mostly exists because IL2CPP rejects IL that MAGIC emits legally. `Editor/MagicPreprocessor.cs` hooks `IPreprocessBuildWithReport`, so on every player build, under either backend, it hands each assembly to `IL2CPPWorkarounds` to be walked with Mono.Cecil. The backend decides what the walk does, not whether it happens: on IL2CPP the workarounds are added, on Mono any left over from a previous IL2CPP build are removed. +The editor side mostly exists because IL2CPP rejects IL that MAGIC emits legally. `Editor/MagicPreprocessor.cs` hooks `IPreprocessBuildWithReport`, so on every player build, under either backend, it hands each assembly to `IL2CPPWorkarounds` to be walked with Mono.Cecil. The backend decides what the walk does, not whether it happens: on IL2CPP the workarounds are added, on Mono any left over from a previous IL2CPP build are removed. Three passes do the work. `EliminateUnreachableInstructions` strips dead IL the AOT linker chokes on. `GenerateGenericWorkaroundMethods` synthesises the generic instantiations IL2CPP's sharing pass needs to see ahead of time. `LinkXmlGenerator` adds `` entries so managed stripping leaves the runtime alone. That rewrite happens in place. `IL2CPPWorkarounds` writes a temporary file, deletes the original and moves the new bytes over it, so after any player build the DLLs in `magic/` are Cecil output rather than compiler output. This is unconditional, and it happens on Mono builds too: reading and writing an assembly with Cecil rebuilds its metadata tables and layout, so the bytes differ even when no instruction changed. Deterministic compilation says nothing about it, because the mutation happens downstream of the compiler. Committing that directory after a build, without restoring it first, ships the wrong bytes. [The bootstrap](./bootstrap.md) covers getting back to a clean state. -A project can also keep ClojureCLR in the editor for hot reload and run MAGIC only in players, which is in fact the default. The package ships both runtimes, MAGIC's fork under `Runtime/magic/` and ClojureCLR under `Runtime/clojure-clr/`, and a define constraint on every shipped DLL decides which set the editor is allowed to load. Consumer setup is [Unity integration](./unity-integration.md#choosing-the-editor-runtime). +A project can also keep ClojureCLR in the editor for hot reload and run MAGIC only in players, which is the default. The package ships both runtimes, MAGIC's fork under `Runtime/magic/` and ClojureCLR under `Runtime/clojure-clr/`, and a define constraint on every shipped DLL decides which set the editor is allowed to load. `Editor/Reload/` holds the hot-reload file watcher, which reloads only when a consumer calls `Poll`. Consumer setup is [Unity integration](./unity-integration.md#choosing-the-editor-runtime). ## The example Unity projects diff --git a/docs/clr-dependency-files.md b/docs/clr-dependency-files.md index 27557590a..a40e65439 100644 --- a/docs/clr-dependency-files.md +++ b/docs/clr-dependency-files.md @@ -100,7 +100,7 @@ Only `test.check` needs an override, because only `test.check` lives somewhere e ### A shipped path reaches the JVM too -The downside of using an alias shows up with CLR-only load paths, the kind [pre-compiled assembly loading](./native-assemblies.md) needs. You have to declare that path in the root `:paths` of `deps.edn`, which means it ships on the JVM too. That is only a problem if you also run the lib on the JVM. +The downside of using an alias shows up with CLR-only load paths, the kind [a library's C# assembly](./native-assemblies.md) needs. You have to declare that path in the root `:paths` of `deps.edn`, which means it ships on the JVM too. That is only a problem if you also run the lib on the JVM. ## Activating aliases, since `nos` takes no alias flag diff --git a/docs/deterministic-compilation.md b/docs/deterministic-compilation.md index dd57ff934..a175151b7 100644 --- a/docs/deterministic-compilation.md +++ b/docs/deterministic-compilation.md @@ -183,6 +183,26 @@ A refresh that fails to compile anything deploys nothing and exits non-zero, so `magic/Clojure.dll` and `magic/Magic.Runtime.dll` are built by csproj, and their csproj stamps a `SourceRevisionId` from `git describe` into the assembly. Their bytes change with every commit by design, and no rebuild reproduces the committed ones. `check-drift` restores those two from HEAD, and maintainers refresh them deliberately. +## Committing an assembly you compiled yourself + +The same property matters one level out. A library that ships a hand-written C# class commits the DLL `csc` produced ([a library's C# assembly](./native-assemblies.md)), and the same rule applies: if the command that built it is not reproducible, every rebuild moves the committed file and the team learns to ignore real changes. + +`-deterministic` covers the first half. Without it Roslyn stamps a fresh module id and timestamp into every build. Mono's older `mcs` produces stable bytes without the flag and accepts it silently, so a build script can pass it always instead of detecting which compiler is installed. + +The `-out:` basename is the trap the flag does not cover. The assembly and module identity come from it, and every C# compiler writes them into the metadata, so renaming the file afterwards does not rename the module inside it. A temp directory is fine, a temp name is not. + +```bash +# stable: the module name is my_lib, the name the committed DLL already has +csc -deterministic -out:/tmp/build/my_lib.dll MyLib.cs +mv /tmp/build/my_lib.dll src_classes/ + +# not stable: the module name is tmp, so the bytes differ from the committed DLL +csc -deterministic -out:/tmp/build/tmp.dll MyLib.cs +mv /tmp/build/tmp.dll src_classes/my_lib.dll +``` + +Every other flag lands in the bytes too, `-optimize+` included, and so does the compiler version. Record the exact command beside the source. + ## Where it came from Staleness has been handled three ways over the years: by convention, where a DLL import named the upstream commit it was built from; by CI, publishing built artifacts so nobody had to rely on their local tree; and by a check inside the repo, first over sources and now over bytes. diff --git a/docs/native-assemblies.md b/docs/native-assemblies.md index 01ea54ac8..e9e121b05 100644 --- a/docs/native-assemblies.md +++ b/docs/native-assemblies.md @@ -1,8 +1,24 @@ -# Loading precompiled native assemblies +# A library's C# assembly -Some libraries ship a hand-written C# class next to their Clojure, compiled ahead of time with `csc` and committed to the repo. +Some libraries ship a hand-written C# class next to their Clojure, compiled with `csc` and committed to the repo. -On the JVM that costs you nothing, because the classpath is how types get resolved: you drop the `.class` files on a `:paths` directory and `:import` finds them. +Three steps, in the order you do them: compile the class, load it into the process so `:import` can see it, then ship it with the Clojure that imports it. + +## Build the DLL with `csc` + +When the C# references Clojure types it has to compile against the same runtime assemblies the host will load, and `nos where` prints the directory they came from, so a build script never guesses at install paths: + +```bash +csc -nologo -deterministic -target:library \ + -reference:$(nos where Clojure.dll) \ + -out:src_classes/my_lib.dll MyLib.cs +``` + +The flags, the compiler version and the `-out:` basename all land in the emitted bytes, so a DLL you commit needs the exact command recorded beside its source or a rebuild months later moves it for no reason anyone can name. [Committing an assembly you compiled yourself](./deterministic-compilation.md#committing-an-assembly-you-compiled-yourself) has the traps. + +## Load it: a loader namespace + +On the JVM loading a class costs you nothing, because the classpath is how types get resolved: you drop the `.class` files in a `:paths` directory and `:import` finds them. The CLR has no classpath for types. `:import` resolves a type name against the assemblies **already loaded into the process**, and it never goes looking for a file. So a fresh process has no idea your DLL exists, and the `:import` fails with a missing type even though the file sits right next to your source. @@ -31,9 +47,7 @@ flowchart TD This is not a MAGIC limitation. [ClojureCLR](https://github.com/clojure/clojure-clr) behaves the same way, and its `assembly-load`, `assembly-load-from` and `assembly-load-file` are the mechanism both runtimes give you. MAGIC inherits them, since its stdlib is a fork of ClojureCLR's. -## Recommended: a dedicated assembly loader namespace - -You have a small namespace that loads the DLL, required by the namespace that imports the types. The tempting shortcut is to hardcode the DLL path, but that breaks the moment someone consumes your library from another directory. Instead scan `CLOJURE_LOAD_PATH`, the environment variable both runtimes fill from the project's `:paths`. MAGIC also has a `*load-paths*` var, but it is MAGIC's own, so a loader reading it breaks under `cljr`. +Write a small namespace that loads the DLL, and require it from the namespace that imports the types. The tempting shortcut is to hardcode the DLL path, but that breaks the moment someone consumes your library from another directory. Instead scan `CLOJURE_LOAD_PATH`, the environment variable both runtimes fill from the project's `:paths`. The loader is CLR-only, so it gets the `.cljr` extension and needs no reader conditionals: @@ -73,17 +87,18 @@ flowchart LR noop --> imp ``` -Three things have to line up: +Several assemblies fold into one loader with a `doseq` over the filenames. The loader is idempotent: `require` runs it once per process, and `assembly-load-from` on a path already loaded returns the cached assembly. + +MAGIC also has a `*load-paths*` var, but it is MAGIC's own, so a loader reading it breaks under `cljr`. Scan the environment variable. -- **The DLL's directory must be on the project's `:paths`.** That is what `nos` and `cljr` turn into `CLOJURE_LOAD_PATH`, so anything outside `:paths` is invisible to the scan. The directory name itself is free, `src_classes` in the examples here. -- **It must be a top-level `:paths` entry, in `deps-clr.edn` if the library ships one.** A `:clr` alias cannot carry it, because a dependency's aliases are never applied: the directory would reach the library's own build and no consumer's. -- **It is recommended to have the DLL named after the namespace prefix of the types inside it**, so `my_lib.MyParser` lives in `my_lib.dll`. It enables MAGIC's hint: when an `:import` fails, MAGIC searches the load path for a DLL whose name matches a namespace prefix of the missing type and points at it. +Note that in Unity, `CLOJURE_LOAD_PATH` is unset in the Editor and in players, so the loader scans nothing and returns, and it has nothing to do there anyway: Unity loads every managed plugin under `Assets/` before any Clojure runs. -Several assemblies fold into one loader with a `doseq` over the filenames, several libraries compose with no coordination from the consumer, and the whole thing is idempotent, since `require` runs a loader once per process and `assembly-load-from` on an already loaded path returns the cached assembly. +### Two rules for the DLL -Note that Unity players are the one place the scan finds nothing, since `CLOJURE_LOAD_PATH` is unset there. That is correct rather than broken, because assemblies under `Assets/Plugins` are already loaded before any Clojure runs. +- **Its directory must be a top-level `:paths` entry, in `deps-clr.edn` if the library ships one.** `:paths` is what `nos` and `cljr` turn into `CLOJURE_LOAD_PATH`, so anything outside it is invisible to the scan, and a `:clr` alias cannot carry it because a dependency's aliases are never applied: the directory would reach the library's own build and no consumer's. +- **Name it after the namespace prefix of the types inside it**, so `my_lib.MyParser` lives in `my_lib.dll`. That is what enables the hint MAGIC prints when an `:import` fails. -### The in-file variant you will see in ClojureCLR libraries +### The in-file variant in ClojureCLR libraries ClojureCLR's own sources do not use a loader namespace. They load inline, in the file that needs the types, and move the `:import` out of the `ns` form so the load can run before it. From [`clojure/clr/io.clj`](https://github.com/clojure/clojure-clr/blob/master/Clojure/Clojure.Source/clojure/clr/io.clj), abridged: @@ -98,9 +113,9 @@ ClojureCLR's own sources do not use a loader namespace. They load inline, in the (import '[System.Net.Sockets Socket NetworkStream]) ``` -It works, and for a single importing namespace there is nothing wrong with it. Note that those cases know their directory up front (`RT/SystemRuntimeDirectory`), which is why they hardcode a path instead of scanning: a DLL shipped inside a library has no such fixed location. +It works, and for a single importing namespace there is nothing wrong with it. Those cases know their directory up front (`RT/SystemRuntimeDirectory`), which is why they hardcode a path instead of scanning: a DLL shipped inside a library has no such fixed location. -## When the loader is missing +### When the loader is missing Nothing warns you in advance. The `:import` fails when it runs, and MAGIC's error names the DLL it spotted on the load path: @@ -110,37 +125,53 @@ System.InvalidOperationException: Could not find type my_lib.MyParser during imp Under `nos build` and `nos test` this surfaces while the namespace compiles, and for a consumer it surfaces at `require`. -MAGIC adds a hint to guide the consumer: it appears when a DLL on the load path matches a namespace prefix of the unresolved type, and the compiler appends it to `Unable to resolve symbol` errors too. +The hint appears when a DLL on the load path matches a namespace prefix of the unresolved type; the compiler appends it to `Unable to resolve symbol` errors too. -## Building the assembly +## Ship it: `nos build` -How you drive `csc` is between you and Microsoft, with one MAGIC-specific part. When the C# references Clojure types it has to compile against the same runtime assemblies the host will load, and `nos where` prints the directory they came from, so a build script never guesses at install paths: +`nos build` compiles your Clojure and copies the DLL you built above. It never compiles C#. -```bash -csc -nologo -deterministic -target:library \ - -reference:$(nos where Clojure.dll) \ - -out:src_classes/my_lib.dll MyLib.cs +```mermaid +flowchart LR + cs["MyLib.cs"] -->|"csc"| dll["my_lib.dll
committed, on a :paths dir"] + clj["my_lib/core.cljc"] + subgraph nb["nos build"] + comp["compiles the
namespaces"] + copy["copies the C# assemblies
the deps ship"] + end + clj --> comp --> out[":out"] + dll --> copy --> co[":csharp-out"] ``` -`-deterministic` is there because the DLL is committed: without it Roslyn stamps a fresh module id and timestamp into every build, so rebuilding unchanged source dirties `git status` and you learn to ignore real changes. Mono's older `mcs` produces stable bytes without the flag and accepts it silently, so a build script can pass it always instead of detecting which compiler is installed. +``` +$ nos build +Compiling my-app.core +Copying C# assembly my_lib.dll +Done. +``` -One trap survives the flag rather than being caused by it. The assembly and module identity come from the `-out:` basename, and every C# compiler writes them into the metadata, so renaming the file afterwards does not rename the module inside it. Compiling to a temp name and moving it into place therefore produces a different DLL than compiling straight to the committed name. A temp directory is fine, a temp name is not. Without `-deterministic` you would never notice, because every rebuild churns anyway. +It hashes the destination first and writes only what changed, so a build that touched no C# makes Unity reimport nothing. Two dependencies shipping the same file name with different content stop the build instead of overwriting each other. -```bash -# stable: the module name is my_lib, the name the committed DLL already has -csc -deterministic -out:/tmp/build/my_lib.dll MyLib.cs -mv /tmp/build/my_lib.dll src_classes/ +Nothing prunes the destination. Drop a library from your deps and the next build names what it left behind, for you to delete: -# not stable: the module name is tmp, so the bytes differ from the committed DLL -csc -deterministic -out:/tmp/build/tmp.dll MyLib.cs -mv /tmp/build/tmp.dll src_classes/my_lib.dll ``` +$ nos build +Compiling my-app.core +No dependency ships old_lib.dll any more; delete it from Assets/Plugins/CSharp +Done. +``` + +The copies land in `:out` next to the compiled Clojure. `:csharp-out` sends them elsewhere, which is what a Unity project wants: [the `nos` CLI](./nos-cli.md#magicedn) defines the key, and [the two plugin folders](./unity-integration.md#the-two-plugin-folders) is why Unity splits them. + +[unity-examples/magic-unity-smoke](../unity-examples/magic-unity-smoke) is the worked example: `csharp-lib/` ships an assembly and a loader, `smoke.csharp` imports its types, and the build sends the assembly to `Assets/Plugins/CSharp`. + +## Under Unity -[Deterministic compilation](./deterministic-compilation.md) covers why this repo cares so much about that property. +Unity treats the DLL as an ordinary managed plugin. Unlike the compiled Clojure assemblies it carries no define constraint, so both Editor runtimes load it and so does every player, Mono and IL2CPP alike. Nothing in it is MAGIC-aware. -## If the assembly ships into a Unity player +### What IL2CPP will not let your C# do -The Clojure side of a library is compiled by MAGIC and already survives IL2CPP. Your C# is not, so it carries the constraints of whatever profile the player uses. Two of them bite: +MAGIC's own output already survives IL2CPP. Your C# carries the constraints of whatever profile the player uses, and two of them bite: - **No runtime codegen.** `System.Reflection.Emit`, `Expression.Compile`, dynamic proxies: all of it works under Mono and throws under IL2CPP, which has no JIT. `nos test` will not catch this, since it runs on Mono. - **Reflection over types nothing references statically** can be stripped from the player. Managed stripping keeps what it can see, so a type resolved only by name at runtime may be gone by the time you ask for it. diff --git a/docs/nos-cli.md b/docs/nos-cli.md index 259cd0122..0669f0655 100644 --- a/docs/nos-cli.md +++ b/docs/nos-cli.md @@ -33,7 +33,7 @@ Arguments are read as EDN, the whole command line at once. When that fails to re Two of the tasks that ship with the host do the work a project needs, which is why most projects need no task file at all. ```bash -nos build # compile the project's namespaces into ./build +nos build # compile the project's namespaces into ./build, with the C# assemblies its deps ship nos test # run its clojure.test suites, exit 1 on any failure or error ``` @@ -61,6 +61,7 @@ It is a map with a `:build` and a `:test` sub-map. Every key has a default, so s | `:exclude` | yes | yes | namespaces to drop from the set | none | | `:flags` | yes | yes | compiler flag overrides | the sets below | | `:out` | yes | | compile output directory | `"build"` | +| `:csharp-out` | yes | | where the C# assemblies a library ships are copied | `:out` | | `:clean?` | yes | | wipe `:out` first | `true` | | `:re` | | yes | regex string scoping the run | the derived namespaces | | `:exclude-vars` | | yes | `deftest` symbols to skip | none | @@ -71,6 +72,7 @@ Note the following: - **`:re`** is matched with `re-matches`, so it has to match a namespace name whole. Write it as a string, since EDN has no regex literal: `"my\\.lib\\..*"`. - **`:exclude-vars`** takes fully-qualified `deftest` symbols. After `require`, the run clears the `:test` metadata on each, so `clojure.test` skips exactly those vars and the rest of their namespace still runs. It is for a handful of platform-specific failures scattered through otherwise-passing namespaces, where excluding the namespace would throw away good tests. A symbol that does not resolve warns rather than failing. +- **`:csharp-out`** separates the copies from the compiled Clojure. `nos build` copies the C# assemblies a library ships ([a library's C# assembly](./native-assemblies.md)) into `:out`, and `:clean?` deletes `:out` before every compile, so Unity imports the plugin fresh and mints it a new GUID each time. A Unity project points this at a directory of its own to hold the GUID still ([the two plugin folders](./unity-integration.md#the-two-plugin-folders)). `:clean?` never touches that directory, so a copy whose library left the deps stays until you delete it; the build names it rather than deleting it. - **`:namespaces`** is the escape hatch for what the derivation cannot see: a single compile root, or a namespace bundled in the repo that no `require` reaches. ## Compiler flags @@ -78,14 +80,14 @@ Note the following: `nos build` compiles under the flags a shipped MAGIC project runs on: ```clojure -{#'*unchecked-math* true +{#'*unchecked-math* false #'*warn-on-reflection* true #'magic.flags/*strongly-typed-invokes* true #'magic.flags/*direct-linking* true #'magic.flags/*elide-meta* false} ``` -- `*unchecked-math*`: integer arithmetic compiles to raw CIL ops, no overflow checks. +- `*unchecked-math*`: Clojure's default. Arithmetic and narrowing casts keep their overflow checks. Set it per namespace where wrapping is wanted. - `*warn-on-reflection*`: interop the compiler cannot resolve statically warns at compile time. - `*strongly-typed-invokes*`: a call whose `Magic.Function` type is statically known lowers to a typed interface call, skipping argument boxing. - `*direct-linking*`: a call to a non-variadic, non-dynamic fn lowers to a direct `invokeStatic`, bypassing the Var. @@ -139,7 +141,7 @@ One difference to know if you keep one: `compile-project` defaults `:clean?` to | `nos print-basis [:alias ...]` | the resolved paths and libs, without compiling | | `nos repl` | a REPL on a warm runtime | -`nos where` is what a build script uses to compile C# against the same runtime the host will load, instead of guessing at install paths ([native assemblies](./native-assemblies.md)). `nos print-basis` is the one to reach for when a namespace is missing and you cannot tell whether the dependency resolved ([declaring CLR dependencies](./clr-dependency-files.md)). +`nos where` is what a build script uses to compile C# against the same runtime the host will load, instead of guessing at install paths ([C# assemblies](./native-assemblies.md)). `nos print-basis` is the one to reach for when a namespace is missing and you cannot tell whether the dependency resolved ([declaring CLR dependencies](./clr-dependency-files.md)). ## What `nos test` cannot catch diff --git a/docs/porting-libraries-to-magic.md b/docs/porting-libraries-to-magic.md index ed859965b..2da0a7065 100644 --- a/docs/porting-libraries-to-magic.md +++ b/docs/porting-libraries-to-magic.md @@ -19,7 +19,7 @@ flowchart LR **4. Run the tests and wire up CI.** The rest of this page. -If the library ships a precompiled C# assembly next to its Clojure, it needs a loader namespace as well: see [loading precompiled native assemblies](./native-assemblies.md). +If the library ships a precompiled C# assembly next to its Clojure, it needs a loader namespace as well: see [a library's C# assembly](./native-assemblies.md). ## Running the tests: two runners, not one diff --git a/docs/unity-integration.md b/docs/unity-integration.md index e813fabeb..418101d49 100644 --- a/docs/unity-integration.md +++ b/docs/unity-integration.md @@ -22,10 +22,12 @@ This is the consumer-side guide. For the package's C# API and install reference, ```clojure ;; magic.edn - {:build {:namespaces [my.game.core] :out "Assets/Plugins/Magic"}} + {:build {:namespaces [my.game.core] + :out "Assets/Plugins/Magic" + :csharp-out "Assets/Plugins/CSharp"}} ``` - A project with custom build/test steps can hand-write a `dotnet.clj` instead; see [the porting guide](./porting-libraries-to-magic.md). + The two output folders are [explained below](#the-two-plugin-folders). A project with custom build/test steps can hand-write a `dotnet.clj` instead; see [the porting guide](./porting-libraries-to-magic.md). 4. **Compile before opening Unity:** @@ -33,7 +35,7 @@ This is the consumer-side guide. For the package's C# API and install reference, nos build ``` - This drops your compiled DLLs into `Assets/Plugins/Magic/`, named by the source extension (`.clj.dll`, `.cljc.dll`, `.cljr.dll`), where Unity loads them. + This drops your compiled DLLs into `Assets/Plugins/Magic/`, named by the source extension (`.clj.dll`, `.cljc.dll`, `.cljr.dll`), where Unity loads them. A dependency's C# assembly is copied into `Assets/Plugins/CSharp/`. 5. **Write a loader, then Play.** Unity doesn't know which DLLs are Clojure or which var is the entry point, so a `MonoBehaviour` has to require and invoke it (pattern: [`SmokeTestRunner.cs`](../unity-examples/magic-unity-smoke/Assets/Scripts/SmokeTestRunner.cs)): @@ -50,6 +52,35 @@ This is the consumer-side guide. For the package's C# API and install reference, 6. **Build a player to exercise the IL2CPP / AOT path.** Editor Play runs under Mono and can't surface IL2CPP-only regressions. The smoke example wires this to a one-click menu; see the [smoke README](../unity-examples/magic-unity-smoke/README.md#run). For CI without Unity, `nos test` runs the Mono-side tests headless but doesn't exercise IL2CPP. +## The two plugin folders + +One folder works: `:csharp-out` defaults to `:out`, MAGIC loads the DLLs the same either way, and neither folder name means anything to the tooling. Split them anyway, because Unity treats a file that keeps being deleted differently from one that stays put. + +```clojure +;; magic.edn +{:build {:namespaces [my.game.core] + :out "Assets/Plugins/Magic" + :csharp-out "Assets/Plugins/CSharp"}} +``` + +| | `Assets/Plugins/Magic/` | `Assets/Plugins/CSharp/` | +|---|---|---| +| Set by | `:out` | `:csharp-out`, defaults to `:out` | +| Holds | your Clojure, compiled | C# assemblies your dependencies ship | +| Written by | `nos build`, compiling your sources | `nos build`, copying files `csc` compiled long before | +| Wiped every build | yes, by `:clean?` | no | +| `.meta` | Unity writes it on import, then the package's hook constrains it | Unity writes it on import, nothing touches it after | +| Define constraint | `!UNITY_EDITOR \|\| MAGIC_RUNTIME_IN_EDITOR`, so the Editor loads it only under MAGIC | none, so it always loads, in both Editor runtimes and every player | +| In git | no, gitignore it | your call, `.meta` files included | + +A library ships only the DLL. Everything else in that table is produced inside your project. + +The "wiped every build" row is the reason. `:clean?` deletes `:out` before every compile, so an assembly sitting there is imported fresh each time and Unity mints it a new GUID. Anything that referenced the old one, a component on a scene object, another importer's settings, points at nothing. A `:csharp-out` of its own is never emptied, so the GUID holds. + +Whether you commit that folder is a second, separate choice, and it comes down to who runs `nos build`. Commit it, `.meta` files included, and a teammate who only opens the Editor (where ClojureCLR compiles your Clojure from source) has the C# plugin without building anything. Gitignore it, like `:out`, if everyone runs `nos build` before opening Unity anyway. + +[A library's C# assembly](./native-assemblies.md) covers the copy step and the loader namespace a library needs for its C# to resolve at compile time. + ## Choosing the Editor runtime The package ships both MAGIC (the default for the player build) and ClojureCLR (the default for the Unity editor); set the Editor runtime via `Project Settings > MAGIC`, or from a script through `Magic.Unity.EditorRuntime` ([package README](../magic-unity/README.md#editor-api)). ClojureCLR is the default because it hot-reloads from source; MAGIC-in-Editor is for reproducing player behaviour before a build. @@ -66,6 +97,7 @@ Note that: - **API Compatibility Level must be `.NET Framework`** (`Project Settings > Player`) for ClojureCLR, as it needs assemblies the .NET Standard profile lacks. - In the default state, `Clojure.Require`/`GetVar` drive ClojureCLR, not MAGIC: the same C# calls, executed by whichever runtime the Editor loaded. ClojureCLR compiles from source, so Editor `Require` needs your Clojure sources on its load path (`CLOJURE_LOAD_PATH`); the DLLs that `nos build` wrote are MAGIC output and stay excluded from the Editor in this state. +- **Hot reload is yours to wire up.** A saved Clojure source can be re-evaluated into the running Editor, but nothing in the package does it: construct a [`Magic.Unity.ClojureReloader`](../magic-unity/README.md#editor-api) over your source roots and poll it from your main-thread loop. ## Shipping your own compiled DLLs diff --git a/docs/writing-cross-platform-clojure.md b/docs/writing-cross-platform-clojure.md index da5be0138..af486b7a2 100644 --- a/docs/writing-cross-platform-clojure.md +++ b/docs/writing-cross-platform-clojure.md @@ -104,7 +104,7 @@ For a reference type, `require` the namespace that defines it and then import th (-> p (update :x + dx) (update :y + dy))) ``` -The `require` is not optional and its position is not either. `:import` resolves a name against the assemblies already loaded in the process, never against the disk, so the namespace defining `Point` has to be loaded first or the compile fails with `Could not find type my.lib.shapes.Point during import`. Clauses run in written order, which is what makes `:require` first sufficient. It is the same rule a precompiled C# assembly runs into, in its harder form, where nothing loads the assembly for you at all ([native assemblies](./native-assemblies.md)). +The `require` is not optional and its position is not either. `:import` resolves a name against the assemblies already loaded in the process, never against the disk, so the namespace defining `Point` has to be loaded first or the compile fails with `Could not find type my.lib.shapes.Point during import`. Clauses run in written order, which is what makes `:require` first sufficient. It is the same rule a precompiled C# assembly runs into, in its harder form, where nothing loads the assembly for you at all ([C# assemblies](./native-assemblies.md)). A hint that cannot resolve is a hard error rather than a warning, so a typo or a missing import fails at compile time. Two hints to avoid: a collection's element type, which has no hint syntax at all, and a map's concrete class, which flips between `PersistentArrayMap` and `PersistentHashMap` with size. Hint the interface `clojure.lang.IPersistentMap` instead. diff --git a/mage/README.md b/mage/README.md index 01aca3272..ad27e91c4 100644 --- a/mage/README.md +++ b/mage/README.md @@ -6,27 +6,23 @@ Quick Example ------------- ```clojure (require '[mage.core :as il]) -(import '[System.Reflection TypeAttributes]) (il/emit! (il/assembly "Example" [(il/module "Example.dll" - [(il/type "ExampleType" TypeAttributes/Public [] System.Object nil + [(il/type "ExampleType" [(il/method "AddIntegers" Int32 [Int32 Int32] [(il/ldarg-1) (il/ldarg-2) (il/add) - (il/ret)])] - [])])])) + (il/ret)])])])])) (.AddIntegers (ExampleType.) 5 6) ;; 11 ``` -`il/type` is the one constructor that has to be spelled out in full: its shorter arities are currently unreachable, so pass attributes, interfaces, supertype, generic parameters, body and custom attributes every time. - Overview -------- MAGE wraps the entire CLR [`System.Reflection.Emit` namespace](https://msdn.microsoft.com/en-us/library/system.reflection.emit(v=vs.110).aspx) in a [gamma](https://github.com/kovasb/gamma)-style symbolic compiler. The goal is a functional, composable, data- and REPL-driven bytecode emission framework for the CLR. A tree of symbolic [MSIL bytecode](https://en.wikipedia.org/wiki/Common_Intermediate_Language) is built as Clojure data, and passed to an `emit!` function that turns it into runnable CLR types in memory. Writing a DLL to disk is the caller's job, on top of that. diff --git a/mage/src/mage/core.clj b/mage/src/mage/core.clj index e5a327e54..363baf7f3 100644 --- a/mage/src/mage/core.clj +++ b/mage/src/mage/core.clj @@ -437,7 +437,7 @@ ([name attributes interfaces body] (type name attributes interfaces System.Object body)) ([name attributes interfaces super body] - (type name attributes interfaces super nil body)) + (type name attributes interfaces super nil body [])) ([name attributes interfaces super generic-parameters body custom-attributes] {::type name ::attributes attributes diff --git a/magic-compiler/dll-sources.edn b/magic-compiler/dll-sources.edn index ae25a1918..cc73b6681 100644 --- a/magic-compiler/dll-sources.edn +++ b/magic-compiler/dll-sources.edn @@ -4,16 +4,16 @@ ;; from. bb check-drift re-records this file and fails if a source ;; changed without its DLL being refreshed and committed. { - clojure.clr.io {:source "magic-compiler/src/stdlib/clojure/clr/io.clj", :sha256 "ef7ea7d3bf39644d06bdc85c68f59ba94e38deabe4db199837b85b2861ed073c"} + clojure.clr.io {:source "magic-compiler/src/stdlib/clojure/clr/io.clj", :sha256 "aaf94e20567eeba46aea272fbb2d83bc4b037aead03a68a04f102ed16482bae7"} clojure.clr.shell {:source "magic-compiler/src/stdlib/clojure/clr/shell.clj", :sha256 "22f036ad1d1dc4965324400d8e9d9663984ac13fc6a483539bf19a332633413a"} - clojure.core {:source "magic-compiler/src/stdlib/clojure/core.clj", :sha256 "2823a838f7e883b1653077388a7d2fcd37c47cf805be530401b4bff378cbc74c"} + clojure.core {:source "magic-compiler/src/stdlib/clojure/core.clj", :sha256 "0adadcd9a9c6b9c5aafaebc48008b30c64f7843ef3e16ea302f0a5908277ecb0"} clojure.core.protocols {:source "magic-compiler/src/stdlib/clojure/core/protocols.clj", :sha256 "38490ad78ad6fabf2d0d965788d6b42b052748ce9d664e24f34033de7e90e378"} clojure.core.reducers {:source "magic-compiler/src/stdlib/clojure/core/reducers.clj", :sha256 "f3f10fef6c91217cf6780c37f358e0f9caecdb106fb5339ac3cdc28b46be950d"} clojure.core.server {:source "magic-compiler/src/stdlib/clojure/core/server.clj", :sha256 "7ba07179820a71f856dfd4c9e1a2a06dbad06eb6e9aeb9bcddeb268959cf82e2"} clojure.core.specs.alpha {:source "magic-compiler/src/stdlib/clojure/core/specs/alpha.clj", :sha256 "41d82e6e56c9165e806f931b0598a05def8388d9b7248d7d431d7e822527928b"} clojure.core_clr {:source "magic-compiler/src/stdlib/clojure/core_clr.clj", :sha256 "371b1553b349138b17e9741e37a134dc4193c621366b1f4b154abcdfa5c69f2e"} clojure.core_deftype {:source "magic-compiler/src/stdlib/clojure/core_deftype.clj", :sha256 "b643bf26b11cc6628116aedddf4569687bb8addff4cf6d0525cd11c4d815b8ad"} - clojure.core_print {:source "magic-compiler/src/stdlib/clojure/core_print.clj", :sha256 "43371f550ebb965398ded47161946b348326c827fa8fde90334b3769618796b8"} + clojure.core_print {:source "magic-compiler/src/stdlib/clojure/core_print.clj", :sha256 "1c1b5346860b8a0f20e945b049f04c2db1443d88f31f1c91ff51c47a1d5a5dae"} clojure.core_proxy {:source "magic-compiler/src/stdlib/clojure/core_proxy.clj", :sha256 "946aeb183fc90cbb1a52dceb325ed81816ed28cea5177ee6dd8b4e801c6cd9c2"} clojure.data {:source "magic-compiler/src/stdlib/clojure/data.clj", :sha256 "e60889339006b57f672c681892710164399ae281d0f37371886b1d9bc73ee8cd"} clojure.datafy {:source "magic-compiler/src/stdlib/clojure/datafy.clj", :sha256 "454d0d43ec60601d031bb62a48ece29f41f8c75648d0da34fe972abce72c56ea"} @@ -25,23 +25,23 @@ clojure.pprint {:source "magic-compiler/src/stdlib/clojure/pprint.clj", :sha256 "a9ff9b336906ba64943de80038fa1a1ddad9aecedb518c16525c5838473ec90f"} clojure.pprint.cl_format {:source "magic-compiler/src/stdlib/clojure/pprint/cl_format.clj", :sha256 "d260fe991b3acc55c40e3c80fef1627e335f472329b9b4bd8d97d03179105ac5"} clojure.pprint.column_writer {:source "magic-compiler/src/stdlib/clojure/pprint/column_writer.clj", :sha256 "bd175e6d656adc69de69e231fd46e6aa00049e740bbe08ea826d11cb6f9cb8d1"} - clojure.pprint.dispatch {:source "magic-compiler/src/stdlib/clojure/pprint/dispatch.clj", :sha256 "2ce913e9b28901167af05d2cb1f05c71194ca541e32496a9943ce5d4c7e91f96"} + clojure.pprint.dispatch {:source "magic-compiler/src/stdlib/clojure/pprint/dispatch.clj", :sha256 "a43f40158467887550864f82af0f3e5f63ebf613e0237f5f80aff10588203a55"} clojure.pprint.pprint_base {:source "magic-compiler/src/stdlib/clojure/pprint/pprint_base.clj", :sha256 "d04a66ac0e985c0b575662a1e47af4da858dcc11e437d10a70b1c047758a3813"} clojure.pprint.pretty_writer {:source "magic-compiler/src/stdlib/clojure/pprint/pretty_writer.clj", :sha256 "8d16c54647cbb97b0ddf4acefbca4a0566933b626ce23a6188a8526437d2496b"} clojure.pprint.print_table {:source "magic-compiler/src/stdlib/clojure/pprint/print_table.clj", :sha256 "9abdefc9a5ceeab44ce91d0c6d4816db7482fb100f3b70be6a88bd9132430a56"} clojure.pprint.utilities {:source "magic-compiler/src/stdlib/clojure/pprint/utilities.clj", :sha256 "85891c3656a9e877232302d2b8c2937bdbb1ebfb14794cd79dc94105dfc20b09"} - clojure.repl {:source "magic-compiler/src/stdlib/clojure/repl.clj", :sha256 "d12ade964f6a9dafe5d4227985c3b4faed14221db9886340abe9582634735c04"} + clojure.repl {:source "magic-compiler/src/stdlib/clojure/repl.clj", :sha256 "f559000e8be08d2491e0daf48bf6ed62b7f522eb04e640a7c56d5acc8ed116a6"} clojure.set {:source "magic-compiler/src/stdlib/clojure/set.clj", :sha256 "1eebc2d19ad2c1a1f7c03fc53b5da9bff7eb00a5355609316f7b0f40897d8f0c"} clojure.spec.alpha {:source "magic-compiler/src/stdlib/clojure/spec/alpha.clj", :sha256 "8ba9e4e0d5826e64734cb58f986a680b9a09d328453c37db0f454cfb4a41ab90"} clojure.spec.gen.alpha {:source "magic-compiler/src/stdlib/clojure/spec/gen/alpha.clj", :sha256 "7529d23186c7622592fce537e119efb914dc54398bdbef45209a6a24ed54ebc4"} clojure.stacktrace {:source "magic-compiler/src/stdlib/clojure/stacktrace.clj", :sha256 "a0065d213e7d7931130956a3c098392d2be0b91b9537b5f13c97bb670beb6b9c"} - clojure.string {:source "magic-compiler/src/stdlib/clojure/string.clj", :sha256 "c34346ea918e1d316d99a178c4f3225a8b0f9a4ff4a6b6f72793807556275e12"} + clojure.string {:source "magic-compiler/src/stdlib/clojure/string.clj", :sha256 "4a1288d7608febb5cf109df44b7d6148cf4308654077e01cb7aab5abbad8612a"} clojure.template {:source "magic-compiler/src/stdlib/clojure/template.clj", :sha256 "b31d7b0bbcab1f44f8970d187aa193640069ca14301daa3d57e8ab47dfeccebb"} clojure.test {:source "magic-compiler/src/stdlib/clojure/test.clj", :sha256 "12f173c4a3c79a0ed81d7b7958f57ba7a8a70441f3d415bd67d198087d820682"} clojure.uuid {:source "magic-compiler/src/stdlib/clojure/uuid.clj", :sha256 "04bb8e17c967f2998796b1c313b9503198b7388956dd496bd6a4a5b168487009"} clojure.walk {:source "magic-compiler/src/stdlib/clojure/walk.clj", :sha256 "3dcae5221cae67f13a69c3ff02de6021c3a87cbc1a6b7ee9bc4fc84c8bef877b"} clojure.zip {:source "magic-compiler/src/stdlib/clojure/zip.clj", :sha256 "b16e741a48aba671d437e94da04839dbeaf2dcff4c0c0c4b1c51f3de4c386276"} - mage.core {:source "mage/src/mage/core.clj", :sha256 "434324953383554a7d7f176076bb4faa9a3bee00233ed98afe0ac0fa56cf7824"} + mage.core {:source "mage/src/mage/core.clj", :sha256 "4f9e615a01401f1e5b24c58e884ab981af80ef9f14afd2fd0e937138c51c8177"} magic.analyzer {:source "magic-compiler/src/magic/analyzer.clj", :sha256 "011b9a71f432d6632c35fd2a29a5462b0ec401714e63926160bddbaf39ab013d"} magic.analyzer.analyze_host_forms {:source "magic-compiler/src/magic/analyzer/analyze_host_forms.clj", :sha256 "1dbbb4208265896fc7c146c1bef63af78b3d097384b1f8ca6ccfc1869e3eb20d"} magic.analyzer.binder {:source "magic-compiler/src/magic/analyzer/binder.clj", :sha256 "bfb28ea9f585912fed8251f7e3ca95df69562af32af8790de9712bd5948536d1"} @@ -49,22 +49,22 @@ magic.analyzer.errors {:source "magic-compiler/src/magic/analyzer/errors.clj", :sha256 "fe53f69897cb8a97d157123cd47330db3b71507aeb63a11582b9572fd8fafa6f"} magic.analyzer.generated_types {:source "magic-compiler/src/magic/analyzer/generated_types.clj", :sha256 "ede9195e51de246d8e96ad2f5071e08e0cf422f9fa965ff31e201b91cb595af0"} magic.analyzer.intrinsics {:source "magic-compiler/src/magic/analyzer/intrinsics.clj", :sha256 "c2bc29eaf2ecccf14d09f88b8ab2f8a96ef1a12563ac8158b2add1240f4d88e3"} - magic.analyzer.literal_reinterpretation {:source "magic-compiler/src/magic/analyzer/literal_reinterpretation.clj", :sha256 "e56eec5f886a9744867dfdc3776b94603f4f39a2e4d14a80420bd6e06c205c86"} + magic.analyzer.literal_reinterpretation {:source "magic-compiler/src/magic/analyzer/literal_reinterpretation.clj", :sha256 "3436dfbef7b929f1b31729cd5b48226298088640202fe6590ea73be8c2056662"} magic.analyzer.loop_bindings {:source "magic-compiler/src/magic/analyzer/loop_bindings.clj", :sha256 "365bb5797f36374fb8237184f6900c2b5d056769b632d04b4109137b5bed3996"} magic.analyzer.novel {:source "magic-compiler/src/magic/analyzer/novel.clj", :sha256 "fa1a06041b8ea03aaa8817aec640cc63472ce9a3f8ce66a7a93438dc8af4f416"} magic.analyzer.reflection {:source "magic-compiler/src/magic/analyzer/reflection.clj", :sha256 "16de58b74c60882c7ee36df8410f30e8d16cec6bb8168d899bcf9a463cb0f8f3"} magic.analyzer.remove_local_children {:source "magic-compiler/src/magic/analyzer/remove_local_children.clj", :sha256 "e5f8971111f3e5b6ad7e9873f1ae1f53a0e8e214da463ed5162b8ffb3948c8d1"} - magic.analyzer.typed_passes {:source "magic-compiler/src/magic/analyzer/typed_passes.clj", :sha256 "dacd9e83cb4c2fc67354c7ef49f6eefc4bc0cd59079b2dbc487cb1b0df2a4e0a"} + magic.analyzer.typed_passes {:source "magic-compiler/src/magic/analyzer/typed_passes.clj", :sha256 "2a99953a27124c360a74dc66a396f6d448fda30eecc61608dd11807f589a34ae"} magic.analyzer.types {:source "magic-compiler/src/magic/analyzer/types.clj", :sha256 "e47a7cf02ac85df1976c0fa23d8389f251f381171aaa98c10b453bea7d6389d9"} magic.analyzer.uniquify {:source "magic-compiler/src/magic/analyzer/uniquify.clj", :sha256 "8efe465c85094d3c1a788403231f5f84dde5c6a169ec66ca897da93bb4794d7a"} magic.analyzer.untyped_passes {:source "magic-compiler/src/magic/analyzer/untyped_passes.clj", :sha256 "19e7f4f2a357f2c265e373dcb1175f336baaf09b5d380744433a4d25f87c4a94"} magic.analyzer.util {:source "magic-compiler/src/magic/analyzer/util.clj", :sha256 "696020a2e8ae86c432ea83d2b783bc1a51fcc999972840889d76564b9a60e385"} magic.api {:source "magic-compiler/src/magic/api.clj", :sha256 "4b78b1f2b9eb22cfd2e6c32fbd2eec81aee87bd489ce48a738b46b54da8ed88c"} - magic.core {:source "magic-compiler/src/magic/core.clj", :sha256 "6107b25f2d94a51ebe26bb42173f4f57ffec1310786419a408077ef7b4fa444e"} + magic.core {:source "magic-compiler/src/magic/core.clj", :sha256 "3752d85fe5cb0288d7f37309325733492d6546699f228550b2e498d4df10028b"} magic.emission {:source "magic-compiler/src/magic/emission.clj", :sha256 "6ad018708a5ef365ced1f762ae1c18bc38ef909e73b24751d599e48a72418bfb"} magic.flags {:source "magic-compiler/src/magic/flags.clj", :sha256 "2557a7162fac069670e7084705608e56780da249be6d95ce80fff985b71c2263"} - magic.interop {:source "magic-compiler/src/magic/interop.clj", :sha256 "17e4d2fbf1a915948b634d9252691c3183e17b25482fdedef1e039a6438cbc98"} - magic.intrinsics {:source "magic-compiler/src/magic/intrinsics.clj", :sha256 "c1e58142c5f9d71fe763a920e6c15d981d69add1f9deeaf45410617dc265d351"} + magic.interop {:source "magic-compiler/src/magic/interop.clj", :sha256 "77d63812bfffc65e41baa5c161ddf6a41b1145a6a11becc8104b6b8897ccbbe2"} + magic.intrinsics {:source "magic-compiler/src/magic/intrinsics.clj", :sha256 "c69d799ae4a07715de4f163e0e00d6b7969409cb487d519553469439aca29fa4"} magic.spells.lift_keywords {:source "magic-compiler/src/magic/spells/lift_keywords.clj", :sha256 "907dc9a166b32a95d237440b5875a186de05e7786390a8fba095382611eaf75e"} magic.spells.lift_vars {:source "magic-compiler/src/magic/spells/lift_vars.clj", :sha256 "1dafcdadb721c083f867e6935b9cb9ea3bdc2e2e7a8f07cd3c0daae9f4293ead"} magic.util {:source "magic-compiler/src/magic/util.clj", :sha256 "62d16cbc45160f486776f6c0c240bf6762846bbcbb59cf1cd45d409defe9c7d7"}} diff --git a/magic-compiler/src/magic/analyzer/literal_reinterpretation.clj b/magic-compiler/src/magic/analyzer/literal_reinterpretation.clj index d9201911e..83acda2e1 100644 --- a/magic-compiler/src/magic/analyzer/literal_reinterpretation.clj +++ b/magic-compiler/src/magic/analyzer/literal_reinterpretation.clj @@ -9,21 +9,28 @@ [types :refer [ast-type numeric integer]]]) (:import [System.Reflection BindingFlags])) -;; TODO idk if this is in the right place -(defn reinterpret-value [val to-type] - (let [v (condp = to-type - Single (Convert/ToSingle val) - Double (Convert/ToDouble val) - Byte (Convert/ToByte val) - SByte (Convert/ToSByte val) - Int16 (Convert/ToInt16 val) - UInt16 (Convert/ToUInt16 val) - Int32 (Convert/ToInt32 val) - UInt32 (Convert/ToUInt32 val) - Int64 (Convert/ToInt64 val) - UInt64 (Convert/ToUInt64 val) - val)] - v)) +(defn reinterpret-value + "Convert rounds (and Mono saturates) where the cast truncates and throws, so + a floating-point or out-of-range literal keeps its type and narrows at + runtime through the checked RT cast instead." + [val to-type] + (if (and (integer to-type) + (or (instance? Single val) (instance? Double val))) + val + (try + (condp = to-type + Single (Convert/ToSingle val) + Double (Convert/ToDouble val) + Byte (Convert/ToByte val) + SByte (Convert/ToSByte val) + Int16 (Convert/ToInt16 val) + UInt16 (Convert/ToUInt16 val) + Int32 (Convert/ToInt32 val) + UInt32 (Convert/ToUInt32 val) + Int64 (Convert/ToInt64 val) + UInt64 (Convert/ToUInt64 val) + val) + (catch OverflowException _ val)))) ;; TODO is this better than e.g. a peephope pass? (defn reinterpret [{:keys [literal? op val] :as ast} to-type] diff --git a/magic-compiler/src/magic/analyzer/typed_passes.clj b/magic-compiler/src/magic/analyzer/typed_passes.clj index d66a18840..c9d01efc2 100644 --- a/magic-compiler/src/magic/analyzer/typed_passes.clj +++ b/magic-compiler/src/magic/analyzer/typed_passes.clj @@ -86,26 +86,65 @@ (throw (ex-info "Invalid type used as volatile field" {:symbol sym :type hint :documentation "https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/volatile"}))))) -(defn analyze-method [{:keys [params name] :as f} candidate-methods type-key this-type explicit-this?] +(defn override-rank + "Total order within a signature group. A base class method sorts first: only + its attributes produce a valid override." + [method] + (str (if (.. method DeclaringType IsInterface) 1 0) + (magic/stable-method-key method))) + +(defn unmatched-method-message + "A return type overload is the one miss a hint resolves, so name the choices. + present-error appends the method name, so the text has to end where it goes." + [by-signature] + (if (and (next by-signature) + (apply = (map second (keys by-signature)))) + (str "Overloaded on return type (" + (string/join ", " (sort (map #(.FullName (nth % 2)) (keys by-signature)))) + "), hint the return type of method ") + "No match binding method")) + +(defn method-name-parts + "The munged name a method spec writes, with its explicit interface prefix + split off. A spec may namespace-qualify the symbol, so drop that first." + [name] (let [name (str name) - name (if (string/includes? name "/") - (subs name (inc (string/last-index-of name "/"))) - name) - name (munge name) + name (munge (if (string/includes? name "/") + (subs name (inc (string/last-index-of name "/"))) + name))] + (if (string/includes? name ".") + (let [last-dot (string/last-index-of name ".")] + [name (subs name 0 last-dot) (subs name (inc last-dot))]) + [name nil name]))) + +(defn candidates-by-signature + [candidate-methods method-name interface-name return-type] + (->> candidate-methods + (filter #(= method-name (.Name %))) + (filter #(or (nil? interface-name) (= interface-name (.. % DeclaringType FullName)))) + (filter #(or (nil? return-type) (= return-type (.ReturnType %)))) + (group-by interop/override-signature))) + +(defn select-overrides + "Every method one emitted override covers. A lone signature group is that + slot; several mean the argument types have to pick between them." + [by-signature arg-types] + (->> (if (= 1 (count by-signature)) + (val (first by-signature)) + (some-> (select-method (map (comp first val) by-signature) arg-types) + interop/override-signature + by-signature)) + (sort-by override-rank u/ordinal-str-compare))) + +(defn analyze-method [{:keys [params name] :as f} candidate-methods type-key this-type explicit-this?] + (let [return-type (types/tag name) + [name interface-name method-name] (method-name-parts name) params* (if explicit-this? (drop 1 params) params) - [interface-name method-name] - (if (string/includes? name ".") - (let [last-dot (string/last-index-of name ".")] - [(subs name 0 last-dot) - (subs name (inc last-dot))]) - [nil name]) - candidate-methods (filter #(= method-name (.Name %)) candidate-methods) - candidate-methods (if interface-name - (filter #(= interface-name (.. % DeclaringType FullName)) candidate-methods) - candidate-methods)] - (if-let [best-method (select-method candidate-methods (map ast-type params*))] + by-signature (candidates-by-signature candidate-methods method-name interface-name return-type) + overrides (select-overrides by-signature (map ast-type params*))] + (if-let [best-method (first overrides)] (let [method-param-types (map #(.ParameterType %) (.GetParameters best-method)) - hinted-param-types (if explicit-this? + hinted-param-types (if explicit-this? (concat [this-type] method-param-types) method-param-types) hinted-params (mapv #(update %1 :form vary-meta assoc :tag %2) params hinted-param-types)] @@ -113,9 +152,12 @@ :name name :params hinted-params :source-method best-method + :override-methods overrides type-key this-type)) - (throw (ex-info "No match binding method" {:name name :params (map ast-type params) :candidates (vec candidate-methods) - :type-key type-key :this-type this-type :explicit-this? explicit-this?}))))) + (throw (ex-info (unmatched-method-message by-signature) + {:name name :params (map ast-type params) :candidates (vec (mapcat val by-signature)) + :type-key type-key :this-type this-type :explicit-this? explicit-this?}))))) + (def make-cctor (memoize (fn [containing-type] diff --git a/magic-compiler/src/magic/core.clj b/magic-compiler/src/magic/core.clj index c881b3ee4..92d31c764 100644 --- a/magic-compiler/src/magic/core.clj +++ b/magic-compiler/src/magic/core.clj @@ -63,8 +63,6 @@ [(il/stloc loc) (il/ldloca loc)]))) -;; TODO overflows? -;; can overflow opcodes replace e.g. RT.intCast? (def intrinsic-conv {Char (il/conv-u2) SByte (il/conv-i1) @@ -81,224 +79,249 @@ (def unsigned-integer #{Byte UInt16 UInt32 UInt64}) -(defn convert-type [from to] - (cond - (= from :magic.analyzer.types/disregard) - nil ; (throw (Exception. "cannot convert from disregarded type")) - (= to :magic.analyzer.types/disregard) - (throw (Exception. "cannot convert to disregarded type")) - - (nil? from) - (recur Object to) - - (nil? to) - (recur from Object) - - ;; do nothing if the types are the same - (= from to) - nil - - (and (types/is-enum? from) (= Object to)) - (il/box from) - - (types/is-enum? from) - (convert-type (Enum/GetUnderlyingType from) to) - - (types/is-enum? to) - (convert-type from (Enum/GetUnderlyingType to)) - - ;; cannot convert nil to value type - (and (nil? from) (types/is-value-type? to)) - (throw (Exception. (str "Cannot convert nil to value type " to))) - - ;; TODO truthiness - (and (types/is-value-type? from) - (= to Boolean)) - [(il/pop) - (il/ldc-i4-1)] - - (and - (= from Object) - (= to Boolean)) - (let [isbool (il/label) - end (il/label)] - [(il/dup) - (il/isinst Boolean) - (il/brtrue isbool) +(def unsigned-primitive + (conj unsigned-integer Char)) + +(def integer-bits + {SByte 8 Byte 8 + Int16 16 UInt16 16 Char 16 + Int32 32 UInt32 32 + Int64 64 UInt64 64}) + +(def rt-cast-method + "clojure.lang.RT cast to each primitive, checked and unchecked. Reference + Clojure converts through these rather than emitting conv.ovf, which throws a + different exception and rejects NaN." + {Char {:checked "charCast" :unchecked "uncheckedCharCast"} + SByte {:checked "sbyteCast" :unchecked "uncheckedSByteCast"} + Byte {:checked "byteCast" :unchecked "uncheckedByteCast"} + Int16 {:checked "shortCast" :unchecked "uncheckedShortCast"} + UInt16 {:checked "ushortCast" :unchecked "uncheckedUShortCast"} + Int32 {:checked "intCast" :unchecked "uncheckedIntCast"} + UInt32 {:checked "uintCast" :unchecked "uncheckedUIntCast"} + Int64 {:checked "longCast" :unchecked "uncheckedLongCast"} + UInt64 {:checked "ulongCast" :unchecked "uncheckedULongCast"} + Single {:checked "floatCast" :unchecked "uncheckedFloatCast"} + Double {:checked "doubleCast" :unchecked "uncheckedDoubleCast"}}) + +(defn rt-cast + "IL calling to's RT cast on a value of type from, nil when RT has no + overload taking from." + [from to kind] + (when-let [names (rt-cast-method to)] + (when-let [method (interop/method RT (names kind) from)] + (il/call method)))) + +(defn value-preserving? + "True when every value of integer primitive from is representable in to." + [from to] + (let [from-bits (integer-bits from) + to-bits (integer-bits to)] + (boolean + (and from-bits to-bits + (if (unsigned-primitive from) + (if (unsigned-primitive to) + (<= from-bits to-bits) + (< from-bits to-bits)) + (and (not (unsigned-primitive to)) + (<= from-bits to-bits))))))) + +(defn checked-conv + "RT cast call for a narrowing primitive conversion, nil when the conversion + cannot lose a value. Only an integer target can overflow." + [from to] + (when (and (integer-bits to) (not (value-preserving? from to))) + (rt-cast from to :checked))) + +(defn convert-type + ([from to] + (convert-type from to false)) + ([from to checked?] + (let [checked-cast (when (and checked? + (types/is-primitive? from) + (types/is-primitive? to)) + (checked-conv from to))] + (cond + (= from :magic.analyzer.types/disregard) + nil ; (throw (Exception. "cannot convert from disregarded type")) + (= to :magic.analyzer.types/disregard) + (throw (Exception. "cannot convert to disregarded type")) + + (nil? from) + (recur Object to checked?) + + (nil? to) + (recur from Object checked?) + + ;; do nothing if the types are the same + (= from to) + nil + + (and (types/is-enum? from) (= Object to)) + (il/box from) + + (types/is-enum? from) + (convert-type (Enum/GetUnderlyingType from) to checked?) + + (types/is-enum? to) + (convert-type from (Enum/GetUnderlyingType to) checked?) + + ;; cannot convert nil to value type + (and (nil? from) (types/is-value-type? to)) + (throw (Exception. (str "Cannot convert nil to value type " to))) + + ;; TODO truthiness + (and (types/is-value-type? from) + (= to Boolean)) + [(il/pop) + (il/ldc-i4-1)] + + (and + (= from Object) + (= to Boolean)) + (let [isbool (il/label) + end (il/label)] + [(il/dup) + (il/isinst Boolean) + (il/brtrue isbool) + (il/ldnull) + (il/cgt-un) + (il/br end) + isbool + (il/unbox-any Boolean) + end]) + + (= to Boolean) + [(il/ldnull) + (il/cgt-un)] + + (and (= from Boolean) + (= to Object)) + (let [istrue (il/label) + end (il/label)] + [(il/brtrue istrue) + (il/ldsfld (interop/field Magic.Constants "False")) + (il/br end) + istrue + (il/ldsfld (interop/field Magic.Constants "True")) + end]) + + (and (= System.Void from) (not (types/is-value-type? to))) (il/ldnull) - (il/cgt-un) - (il/br end) - isbool - (il/unbox-any Boolean) - end]) - - (= to Boolean) - [(il/ldnull) - (il/cgt-un)] - - (and (= from Boolean) - (= to Object)) - (let [istrue (il/label) - end (il/label)] - [(il/brtrue istrue) - (il/ldsfld (interop/field Magic.Constants "False")) - (il/br end) - istrue - (il/ldsfld (interop/field Magic.Constants "True")) - end]) - - (and (= System.Void from) (not (types/is-value-type? to))) - (il/ldnull) - - (and (= System.Void to) (not= System.Void from)) - (il/pop) - - (and (= System.Void from) (types/is-value-type? to)) - (throw (Exception. (str "Cannot convert void to value type " to))) - - ;; use user defined implicit conversion if it exists - (interop/method to "op_Implicit" from) - (il/call (interop/method to "op_Implicit" from)) - - ;; use user defined explicit conversion if it exists - (interop/method to "op_Explicit" from) - (il/call (interop/method to "op_Explicit" from)) - - ;; widening to 8 bytes: extension follows source signedness, not the target - (and (types/integer-type? from) - (#{Int64 UInt64} to) - (not (#{Int64 UInt64} from))) - (if (unsigned-integer from) (il/conv-u8) (il/conv-i8)) - - ;; use intrinsic conv opcodes from primitive to primitive - (and (types/is-primitive? from) (types/is-primitive? to)) - (intrinsic-conv to) - - ;; box valuetypes to objects - (and (types/is-value-type? from) (= to Object)) - (il/box from) - - ;; RT casts - (and (= from Object) (= to Single)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedFloatCast" from) - (interop/method RT "floatCast" from))) - (and (= from Object) (= to Double)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedDoubleCast" from) - (interop/method RT "doubleCast" from))) - (and (= from Object) (= to Int32)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedIntCast" from) - (interop/method RT "intCast" from))) - (and (= from Object) (= to Int64)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedLongCast" from) - (interop/method RT "longCast" from))) - (and (= from Object) (= to Byte)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedByteCast" from) - (interop/method RT "byteCast" from))) - (and (= from Object) (= to SByte)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedSByteCast" from) - (interop/method RT "sbyteCast" from))) - (and (= from Object) (= to Int16)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedShortCast" from) - (interop/method RT "shortCast" from))) - (and (= from Object) (= to UInt16)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedUShortCast" from) - (interop/method RT "ushortCast" from))) - (and (= from Object) (= to UInt32)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedUIntCast" from) - (interop/method RT "uintCast" from))) - (and (= from Object) (= to UInt64)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedULongCast" from) - (interop/method RT "ulongCast" from))) - (and (= from Object) (= to Char)) - (il/call (if *unchecked-math* - (interop/method RT "uncheckedCharCast" from) - (interop/method RT "charCast" from))) - - ;; unbox objects to valuetypes - ;; TODO this will throw an exception of the object - ;; does not have the exact runtime type of the valuetype - ;; ie it does not perform a conversion like the above clauses - (and (= from Object) (types/is-value-type? to)) - (il/unbox-any to) - - ;; castclass if to is a subclass of from - (.IsSubclassOf to from) - (il/castclass to) - - ;; do nothing if converting to super class - (.IsSubclassOf from to) - nil - - (and (types/is-value-type? from) - (.IsAssignableFrom to from)) - [(reference-to-type from) - (il/box from)] + + (and (= System.Void to) (not= System.Void from)) + (il/pop) + + (and (= System.Void from) (types/is-value-type? to)) + (throw (Exception. (str "Cannot convert void to value type " to))) + + ;; use user defined implicit conversion if it exists + (interop/method to "op_Implicit" from) + (il/call (interop/method to "op_Implicit" from)) + + ;; use user defined explicit conversion if it exists + (interop/method to "op_Explicit" from) + (il/call (interop/method to "op_Explicit" from)) + + checked-cast + checked-cast + + ;; widening to 8 bytes: extension follows source signedness, not the target + (and (types/integer-type? from) + (#{Int64 UInt64} to) + (not (#{Int64 UInt64} from))) + (if (unsigned-integer from) (il/conv-u8) (il/conv-i8)) + + ;; use intrinsic conv opcodes from primitive to primitive + (and (types/is-primitive? from) (types/is-primitive? to)) + (intrinsic-conv to) + + ;; box valuetypes to objects + (and (types/is-value-type? from) (= to Object)) + (il/box from) + + ;; RT casts + (and (= from Object) (rt-cast-method to)) + (rt-cast Object to (if *unchecked-math* :unchecked :checked)) + + ;; unbox objects to valuetypes + ;; TODO this will throw an exception of the object + ;; does not have the exact runtime type of the valuetype + ;; ie it does not perform a conversion like the above clauses + (and (= from Object) (types/is-value-type? to)) + (il/unbox-any to) + + ;; castclass if to is a subclass of from + (.IsSubclassOf to from) + (il/castclass to) + + ;; do nothing if converting to super class + (.IsSubclassOf from to) + nil + + (and (types/is-value-type? from) + (.IsAssignableFrom to from)) + [(reference-to-type from) + (il/box from)] - ;; (.IsAssignableFrom to from) - ;; nil - - (.IsAssignableFrom from to) - (il/castclass to) - - ;; emit ToString when possible - (= to String) - [(reference-to-type from) - ((if (types/is-value-type? from) - il/call - il/callvirt) - (interop/method from "ToString"))] - - (and (not (types/is-value-type? from)) - (not (types/is-value-type? to))) - (il/castclass to) - - (isa? from IConvertible) - (let [method (cond - (= to Double) "ToDouble" - (= to Single) "ToSingle" - (= to Boolean) "ToBoolean" - (= to Byte) "ToByte" - (= to Char) "ToChar" - (= to DateTime) "ToDateTime" - (= to Decimal) "ToDecimal" - (= to Int16) "ToInt16" - (= to Int32) "ToInt32" - (= to Int64) "ToInt64" - (= to UInt16) "ToUInt16" - (= to UInt32) "ToUInt32" - (= to UInt64) "ToUInt64" - (= to Byte) "ToByte" - (= to SByte) "ToSByte" - (= to String) "ToString")] - [(il/castclass IConvertible) - (il/ldnull) - (il/callvirt (magic.interop/method IConvertible method IFormatProvider))]) + ;; (.IsAssignableFrom to from) + ;; nil + + (.IsAssignableFrom from to) + (il/castclass to) + + ;; emit ToString when possible + (= to String) + [(reference-to-type from) + ((if (types/is-value-type? from) + il/call + il/callvirt) + (interop/method from "ToString"))] + + (and (not (types/is-value-type? from)) + (not (types/is-value-type? to))) + (il/castclass to) + + (isa? from IConvertible) + (let [method (cond + (= to Double) "ToDouble" + (= to Single) "ToSingle" + (= to Boolean) "ToBoolean" + (= to Byte) "ToByte" + (= to Char) "ToChar" + (= to DateTime) "ToDateTime" + (= to Decimal) "ToDecimal" + (= to Int16) "ToInt16" + (= to Int32) "ToInt32" + (= to Int64) "ToInt64" + (= to UInt16) "ToUInt16" + (= to UInt32) "ToUInt32" + (= to UInt64) "ToUInt64" + (= to Byte) "ToByte" + (= to SByte) "ToSByte" + (= to String) "ToString")] + [(il/castclass IConvertible) + (il/ldnull) + (il/callvirt (magic.interop/method IConvertible method IFormatProvider))]) - :else - (throw (Exception. (str "Cannot convert " from " to " to))))) - -(defn convert [ast to] - (when-not (:op ast) - (throw (Exception. (str "refactor, first arg to convert needs to be an ast map, got " ast)))) - (cond - (and (= :const (:op ast)) - (= (ast-type ast) Boolean) - (= Object to)) - [(il/pop) - (if (-> ast :val) - (il/ldsfld (interop/field Magic.Constants "True")) - (il/ldsfld (interop/field Magic.Constants "False")))] - :else (convert-type (ast-type ast) to))) + :else + (throw (Exception. (str "Cannot convert " from " to " to))))))) + +(defn convert + ([ast to] + (convert ast to false)) + ([ast to checked?] + (when-not (:op ast) + (throw (Exception. (str "refactor, first arg to convert needs to be an ast map, got " ast)))) + (cond + (and (= :const (:op ast)) + (= (ast-type ast) Boolean) + (= Object to)) + [(il/pop) + (if (-> ast :val) + (il/ldsfld (interop/field Magic.Constants "True")) + (il/ldsfld (interop/field Magic.Constants "False")))] + :else (convert-type (ast-type ast) to checked?)))) (defmulti load-constant type) @@ -884,13 +907,12 @@ (defn static-method-compiler "Symbolic bytecode for static methods" [{:keys [method args] :as ast} compilers] - (let [arg-types (map ast-type args)] - [(interleave - (map #(compile % compilers) args) - (mapv convert - args - (interop/parameter-types method))) - (il/call method)])) + [(interleave + (map #(compile % compilers) args) + (mapv #(convert %1 %2 (not *unchecked-math*)) + args + (interop/parameter-types method))) + (il/call method)]) (defn instance-method-compiler "Symbolic bytecode for instance methods" @@ -904,7 +926,7 @@ (compile target compilers)) (interleave (mapv #(compile % compilers) args) - (mapv convert args (interop/parameter-types method))) + (mapv #(convert %1 %2 (not *unchecked-math*)) args (interop/parameter-types method))) (cond non-virtual? (il/call method) @@ -937,7 +959,7 @@ constructor)] [(interleave (map #(compile % compilers) args) - (map convert + (map #(convert %1 %2 (not *unchecked-math*)) args (interop/parameter-types constructor))) (il/newobj constructor)])) @@ -1135,13 +1157,18 @@ (apply interop/method IFn "invoke" (concat (repeat 20 Object) [System.Object|[]|]))) (defn ifn-invoke-compiler [{:keys [args] :as ast} compilers] - (let [positional-args (take 20 args) + (let [callee (:fn ast) + positional-args (take 20 args) rest-args (drop 20 args) invoke-method (if (empty? rest-args) (ifn-invoke-methods (count args)) variadic-ifn-invoke-method)] - [(il/castclass IFn) + ;; castclass needs an object reference, so a value-typed callee is boxed + ;; first or the method it lands in fails verification + [(when (types/is-value-type? (ast-type callee)) + (convert callee Object)) + (il/castclass IFn) (interleave (map #(compile % compilers) positional-args) (map #(convert % Object) positional-args)) @@ -1803,6 +1830,17 @@ (sort-by (comp stable-method-key key) u/ordinal-str-compare) (mapv val))) +(defn default-overrides + "Throwing defaults for the slots no written method covers, one per slot." + [abstract-methods written attributes] + (->> abstract-methods + (remove (into #{} (mapcat :override-methods written))) + (group-by interop/override-signature) + vals + (map #(first (sort-by stable-method-key u/ordinal-str-compare %))) + (map #(vector % (default-override-method % attributes))) + (into {}))) + (defn compile-proxy-type [{:keys [args super interfaces closed-overs fns proxy-type] :as ast} compilers] (when-not (.IsCreated proxy-type) (let [super-override (enum-or MethodAttributes/Public MethodAttributes/Virtual) @@ -1811,17 +1849,8 @@ interfaces (conj interfaces clojure.lang.IProxy) ;; need to gather *all* interfaces this type effectively supports ifaces* (into #{} (concat interfaces (mapcat #(.GetInterfaces %) interfaces))) - iface-methods - (->> ifaces* - (mapcat (fn [iface] - (map - #(vector % (default-override-method % iface-override)) - (all-abstract-methods iface)))) - (into {})) - abstract-methods - (into {} - (map #(vector % (default-override-method % super-override)) - (all-abstract-methods super))) + iface-methods (default-overrides (mapcat all-abstract-methods ifaces*) fns iface-override) + abstract-methods (default-overrides (all-abstract-methods super) fns super-override) closed-over-field-map (reduce-kv (fn [m k v] @@ -2044,13 +2073,7 @@ ifaces* (disj (into #{} (concat interfaces (mapcat #(.GetInterfaces %) interfaces))) clojure.lang.IObj clojure.lang.IMeta) - iface-methods - (->> ifaces* - (mapcat (fn [iface] - (map - #(vector % (default-override-method % iface-override)) - (all-abstract-methods iface)))) - (into {})) + iface-methods (default-overrides (mapcat all-abstract-methods ifaces*) methods iface-override) provided-methods (into {} (map (fn [m] [(:source-method m) (compile m specialized-compilers)]) methods)) methods* (merge iface-methods provided-methods)] @@ -2115,13 +2138,7 @@ (let [super-override (enum-or MethodAttributes/Public MethodAttributes/Virtual) iface-override (enum-or super-override MethodAttributes/Final MethodAttributes/NewSlot) ifaces* (into #{} (concat implements (mapcat #(.GetInterfaces %) implements))) - iface-methods - (->> ifaces* - (mapcat (fn [iface] - (map - #(vector % (default-override-method % iface-override)) - (all-abstract-methods iface)))) - (into {})) + iface-methods (default-overrides (mapcat all-abstract-methods ifaces*) methods iface-override) defrecord? (.IsAssignableFrom clojure.lang.IRecord deftype-type) fieldinfos (.GetFields deftype-type) fieldinfos-set (into #{} (.GetFields deftype-type)) diff --git a/magic-compiler/src/magic/interop.clj b/magic-compiler/src/magic/interop.clj index 507acb536..d7f778ce7 100644 --- a/magic-compiler/src/magic/interop.clj +++ b/magic-compiler/src/magic/interop.clj @@ -43,6 +43,13 @@ [method] (map #(.ParameterType %) (parameters method))) +(defn override-signature + "Methods sharing this key occupy one slot, so one override implements them all. + A derived interface that redeclares an inherited member yields one MethodInfo + per declaring type; only a differing return type makes a second slot." + [method] + [(.Name method) (parameter-types method) (.ReturnType method)]) + (def field (memoize (fn [type name] (.GetField type name)))) diff --git a/magic-compiler/src/magic/intrinsics.clj b/magic-compiler/src/magic/intrinsics.clj index df388268d..64f6ce766 100644 --- a/magic-compiler/src/magic/intrinsics.clj +++ b/magic-compiler/src/magic/intrinsics.clj @@ -95,11 +95,11 @@ [{:keys [args]} type compilers] (let [arg (reinterpret (first args) type)] [(magic/compile arg compilers) - (magic/convert arg type)])) + (magic/convert arg type (not *unchecked-math*))])) ;; Vars without an :inline (sbyte, uint, ulong, ushort) need no method entry. ;; The unchecked casts stay unkeyed: conversion-compiler emits the checked -;; cast for Object sources, which would turn their wrapping into throwing. +;; cast, which would turn their wrapping into throwing. (def conversions {'clojure.core/float [Single "floatCast"] 'clojure.core/double [Double "doubleCast"] @@ -414,8 +414,7 @@ index-arg (reinterpret index-arg Int32)] [(magic/compile array-arg compilers) (magic/compile index-arg compilers) - ;; TODO make sure this is conv.ovf - (magic/convert-type (ast-type index-arg) Int32) + (magic/convert-type (ast-type index-arg) Int32 (not *unchecked-math*)) (magic/load-element type)])) [RT "aget" 2]) @@ -437,10 +436,9 @@ statement? (magic/statement? ast)] [(magic/compile array-arg compilers) (magic/compile index-arg compilers) - ;; TODO make sure this is conv.ovf - (magic/convert-type (ast-type index-arg) Int32) + (magic/convert-type (ast-type index-arg) Int32 (not *unchecked-math*)) (magic/compile value-arg compilers) - (magic/convert-type (ast-type value-arg) type) + (magic/convert-type (ast-type value-arg) type (not *unchecked-math*)) (when-not statement? [(il/dup) (il/stloc val-return)]) @@ -462,7 +460,7 @@ (if-not (= index-arg' index-arg) (magic/compile index-arg' compilers) [(magic/compile index-arg compilers) - (magic/convert index-arg Int32)]) + (magic/convert index-arg Int32 (not *unchecked-math*))]) (if value-type? (il/ldelem type) (il/ldelem-ref))])) @@ -540,7 +538,7 @@ (fn intrinsic-make-array-compiler [{[type-arg len-arg] :args} type compilers] [(magic/compile len-arg compilers) - (magic/convert len-arg Int32) + (magic/convert len-arg Int32 (not *unchecked-math*)) (il/newarr (:val type-arg))])) (defintrinsic clojure.core/enum-or diff --git a/magic-compiler/src/stdlib/clojure/clr/io.clj b/magic-compiler/src/stdlib/clojure/clr/io.clj index 794b9ef6e..3e32163a8 100644 --- a/magic-compiler/src/stdlib/clojure/clr/io.clj +++ b/magic-compiler/src/stdlib/clojure/clr/io.clj @@ -208,9 +208,10 @@ (defn- ^FileMode file-mode [mode opts] (or (:file-mode opts) - (if (= mode :read) - FileMode/Open - FileMode/OpenOrCreate))) + (cond + (= mode :read) FileMode/Open + (:append opts) FileMode/Append + :else FileMode/Create))) (defn- ^FileShare file-share [opts] (or (:file-share opts) FileShare/None)) diff --git a/magic-compiler/src/stdlib/clojure/core.clj b/magic-compiler/src/stdlib/clojure/core.clj index 7698cf440..436d3e36e 100644 --- a/magic-compiler/src/stdlib/clojure/core.clj +++ b/magic-compiler/src/stdlib/clojure/core.clj @@ -286,7 +286,7 @@ (if (clojure.lang.Util/equals nil (maybe-special-tag tag)) ;;; clojure.lang.Compiler$HostExpr (let [c (clojure.lang.RT/classForName tag-name)] ;;; clojure.lang.Compiler$HostExpr maybeClass (if c - (with-meta argvec (assoc m :tag (clojure.lang.Symbol/intern (.Name c)))) ;;; .getName + (with-meta argvec (assoc m :tag (clojure.lang.Symbol/intern (.FullName c)))) ;;; .getName argvec)) argvec) argvec) @@ -3156,7 +3156,7 @@ (sort compare coll)) ([comp coll] ;;; We can't pass in a Comparator directly at this point, only a ClojureRuntimeDelegate : [^java.util.Comparator comp coll] (if (seq coll) - (. clojure.lang.RT (SortedSeq (seq coll) comp)) + (with-meta (. clojure.lang.RT (SortedSeq (seq coll) comp)) (meta coll)) ()))) (defn sort-by diff --git a/magic-compiler/src/stdlib/clojure/core_print.clj b/magic-compiler/src/stdlib/clojure/core_print.clj index da3f36eda..e7b04b5e6 100644 --- a/magic-compiler/src/stdlib/clojure/core_print.clj +++ b/magic-compiler/src/stdlib/clojure/core_print.clj @@ -106,9 +106,11 @@ (print-meta o w)) (.Write w "#object[") (let [c (class o)] + ;; NOTE: dead code on the CLR -- arrays dispatch to the ICollection print-method + ;; below and print as seqs. Kept for parity (if (.IsArray c) ;;; .isArray - (print-method (.Name c) w) ;;; .getName - (.Write w (.Name c)))) ;;; .getName + (print-method (.FullName c) w) ;;; .getName + (.Write w (.FullName c)))) ;;; .getName (.Write w " ") (.Write w (format "0x%x " (System.Runtime.CompilerServices.RuntimeHelpers/GetHashCode o))) ;;; (System/identityHashCode o) (print-method rep w) @@ -265,14 +267,14 @@ (print-meta v w) (print-sequential "[" pr-on " " "]" v w)) -(defn- print-prefix-map [prefix m print-one w] +(defn- print-prefix-map [prefix kvs print-one w] (print-sequential (str prefix "{") - (fn [e ^System.IO.TextWriter w] - (do (print-one (key e) w) (.Write w \space) (print-one (val e) w))) + (fn [[k v] ^System.IO.TextWriter w] + (do (print-one k w) (.Write w \space) (print-one v w))) ", " "}" - (seq m) w)) + kvs w)) (defn- print-map [m print-one w] (print-prefix-map nil m print-one w)) @@ -284,25 +286,25 @@ (keyword nil (name named)))) (defn- lift-ns - "Returns [lifted-ns lifted-map] or nil if m can't be lifted." + "Returns [lifted-ns lifted-kvs] or nil if m can't be lifted." [m] (when *print-namespace-maps* (loop [ns nil [[k v :as entry] & entries] (seq m) - lm {}] + kvs []] (if entry - (when (or (keyword? k) (symbol? k)) + (when (qualified-ident? k) (if ns (when (= ns (namespace k)) - (recur ns entries (assoc lm (strip-ns k) v))) + (recur ns entries (conj kvs [(strip-ns k) v]))) (when-let [new-ns (namespace k)] - (recur new-ns entries (assoc lm (strip-ns k) v))))) - [ns (apply conj (empty m) lm)])))) + (recur new-ns entries (conj kvs [(strip-ns k) v]))))) + [ns kvs])))) (defmethod print-method clojure.lang.IPersistentMap [m, ^System.IO.TextWriter w] - (let [[ns lift-map] (lift-ns m)] + (let [[ns lift-kvs] (lift-ns m)] (if ns - (print-prefix-map (str "#:" ns) lift-map pr-on w) + (print-prefix-map (str "#:" ns) lift-kvs pr-on w) (print-map m pr-on w)))) (defmethod print-dup System.Collections.IDictionary [m, ^System.IO.TextWriter w] ;;; java.util.Map @@ -578,6 +580,7 @@ print-via #(do (.Write w "{:type ") (print-method (:type %) w) (.Write w "\n :message ") + (print-method (:message %) w) (when-let [data (:data %)] (.Write w "\n :data ") (print-method data w)) diff --git a/magic-compiler/src/stdlib/clojure/pprint/dispatch.clj b/magic-compiler/src/stdlib/clojure/pprint/dispatch.clj index 021694325..bf26b9b6d 100644 --- a/magic-compiler/src/stdlib/clojure/pprint/dispatch.clj +++ b/magic-compiler/src/stdlib/clojure/pprint/dispatch.clj @@ -62,8 +62,19 @@ ;;; are a little easier on the stack. (Or, do "real" compilation, a ;;; la Common Lisp) +(declare pprint-map) + +(defn- pprint-meta [obj] + (when *print-meta* + (when-let [m (meta obj)] + (.Write ^System.IO.TextWriter *out* "^") ;;; ^java.io.Writer + (pprint-map m) + (.Write ^System.IO.TextWriter *out* " ") ;;; ^java.io.Writer + (pprint-newline :linear)))) + ;;; (def pprint-simple-list (formatter-out "~:<~@{~w~^ ~_~}~:>")) (defn- pprint-simple-list [alis] + (pprint-meta alis) (pprint-logical-block :prefix "(" :suffix ")" (print-length-loop [alis (seq alis)] (when alis @@ -79,6 +90,7 @@ ;;; (def pprint-vector (formatter-out "~<[~;~@{~w~^ ~_~}~;]~:>")) (defn- pprint-vector [avec] + (pprint-meta avec) (pprint-logical-block :prefix "[" :suffix "]" (print-length-loop [aseq (seq avec)] (when aseq @@ -92,6 +104,7 @@ ;;; (def pprint-map (formatter-out "~<{~;~@{~<~w~^ ~_~w~:>~^, ~_~}~;}~:>")) (defn- pprint-map [amap] + (pprint-meta amap) (let [[ns lift-map] (when (not (record? amap)) (#'clojure.core/lift-ns amap)) amap (or lift-map amap) @@ -110,7 +123,17 @@ (pprint-newline :linear) (recur (next aseq)))))))) -(def ^{:private true} pprint-set (formatter-out "~<#{~;~@{~w~^ ~:_~}~;}~:>")) +;;; (def ^{:private true} pprint-set (formatter-out "~<#{~;~@{~w~^ ~:_~}~;}~:>")) +(defn- pprint-set [aset] + (pprint-meta aset) + (pprint-logical-block :prefix "#{" :suffix "}" + (print-length-loop [aseq (seq aset)] + (when aseq + (write-out (first aseq)) + (when (next aseq) + (.Write ^System.IO.TextWriter *out* " ") ;;; ^java.io.Writer + (pprint-newline :linear) + (recur (next aseq))))))) (def ^{:private true} type-map {"core$future_call" "Future", diff --git a/magic-compiler/src/stdlib/clojure/repl.clj b/magic-compiler/src/stdlib/clojure/repl.clj index 89c5bb73a..33109414e 100644 --- a/magic-compiler/src/stdlib/clojure/repl.clj +++ b/magic-compiler/src/stdlib/clojure/repl.clj @@ -94,18 +94,17 @@ itself (not its value) is returned. The reader macro #'x expands to (var x)."}}) (prn arglists)) (cond special-form - (do - (println "Special Form") - (println " " doc) - (if (contains? m :url) - (when url - (println (str "\n Please see http://clojure.org/" url))) - (println (str "\n Please see http://clojure.org/special_forms#" nm)))) + (println "Special Form") macro (println "Macro") spec (println "Spec")) (when doc (println " " doc)) + (when special-form + (if (contains? m :url) + (when url + (println (str "\n Please see http://clojure.org/" url))) + (println (str "\n Please see http://clojure.org/special_forms#" nm)))) (when n (when-let [fnspec (spec/get-spec (symbol (str (ns-name n)) (name nm)))] (println "Spec") diff --git a/magic-compiler/src/stdlib/clojure/string.clj b/magic-compiler/src/stdlib/clojure/string.clj index 0032683ff..e5136c0a7 100644 --- a/magic-compiler/src/stdlib/clojure/string.clj +++ b/magic-compiler/src/stdlib/clojure/string.clj @@ -218,15 +218,30 @@ Design notes for clojure.string: (defn split "Splits string on a regular expression. Optional argument limit is - the maximum number of splits. Not lazy. Returns vector of the splits." + the maximum number of splits. Not lazy. Returns vector of the splits. + Trailing empty strings are not returned - pass limit of -1 to return all." {:added "1.2"} ([^String s ^Regex re] ;;; ^Pattern - (LazilyPersistentVector/createOwning (.Split re s))) ;;; .split + (split s re 0)) ([^String s ^Regex re limit] ;;; ^Pattern - (LazilyPersistentVector/createOwning (.Split re s limit)))) ;;; .split + ;; Java's limit rules and its zero-width-at-0 rule are applied here, + ;; because .NET's Regex.Split has none of them. + (if (pos? limit) + (LazilyPersistentVector/createOwning (.Split re s limit)) ;;; .split + (let [m (.Match re s) + split (LazilyPersistentVector/createOwning (.Split re s)) ;;; .split + parts (if (and (.Success m) (zero? (.Index m)) (zero? (.Length m))) + (subvec split 1) + split)] + (if (or (neg? limit) (= 1 (count parts))) + parts + (loop [parts parts] + (if (and (seq parts) (= "" (peek parts))) + (recur (pop parts)) + parts))))))) (defn split-lines - "Splits s on \\n or \\r\\n." + "Splits s on \\n or \\r\\n. Trailing empty lines are not returned." {:added "1.2"} [^String s] (split s #"\r?\n")) diff --git a/magic-compiler/test.clj b/magic-compiler/test.clj index db677e2aa..30a6573f9 100644 --- a/magic-compiler/test.clj +++ b/magic-compiler/test.clj @@ -11,6 +11,7 @@ magic.test.special magic.test.proxy magic.test.reify + magic.test.deftype magic.test.fn magic.test.letfn magic.test.pipeline @@ -23,7 +24,8 @@ magic.test.flags magic.test.protocol magic.test.errors - magic.test.load) + magic.test.load + magic.test.mage) (:use clojure.test)) (defn- check-summary! @@ -49,6 +51,7 @@ 'magic.test.dynamic 'magic.test.proxy 'magic.test.reify + 'magic.test.deftype 'magic.test.fn 'magic.test.letfn 'magic.test.pipeline @@ -61,7 +64,8 @@ 'magic.test.flags 'magic.test.protocol 'magic.test.errors - 'magic.test.load))) + 'magic.test.load + 'magic.test.mage))) (defn run [& namespaces] (check-summary! (apply run-tests namespaces))) \ No newline at end of file diff --git a/magic-compiler/test/magic/test/deftype.clj b/magic-compiler/test/magic/test/deftype.clj new file mode 100644 index 000000000..15164c1b9 --- /dev/null +++ b/magic-compiler/test/magic/test/deftype.clj @@ -0,0 +1,93 @@ +(ns magic.test.deftype + (:require [clojure.test :refer [deftest is]] + [magic.api :as m])) + +(deftest inherited-slot-covered-by-one-method + ;; Counted, IPersistentCollection and IPersistentMap each declare int count(). + ;; RT.count calls the Counted slot, so one written count has to fill all three. + (is (= 2 (m/eval '(do (deftype CountBox [m] + clojure.lang.IPersistentMap + (count [_] (count m)) + (seq [_] (seq m))) + (count (CountBox. {:a 1 :b 2}))))))) + +(deftest return-type-hint-selects-overload + (is (= [true true 1] + (m/eval '(do (deftype ConsBox [m] + clojure.lang.IPersistentMap + (count [_] (count m)) + (^clojure.lang.IPersistentCollection cons [this _] this) + (^clojure.lang.IPersistentMap assoc [this _ _] this) + (^clojure.lang.Associative assoc [this _ _] this) + (seq [_] (seq m))) + (let [b (ConsBox. {:a 1})] + [(identical? b (conj b [:c 3])) + (identical? b (assoc b :c 3)) + (count b)])))))) + +(deftest unwritten-slot-emits-one-default + (is (= 1 (m/eval '(do (deftype PlainBox [m] + clojure.lang.IPersistentMap + (seq [_] (seq m))) + (->> (.GetMethods PlainBox (enum-or System.Reflection.BindingFlags/Instance + System.Reflection.BindingFlags/Public + System.Reflection.BindingFlags/DeclaredOnly)) + (filter #(= "count" (.Name %))) + count)))))) + +(deftest unhinted-overload-names-the-return-types + (is (thrown-with-msg? + clojure.lang.ClojureException #"Overloaded on return type" + (m/eval '(deftype AmbiguousBox [m] + clojure.lang.IPersistentMap + (cons [this _] this) + (seq [_] (seq m))))))) + +(deftest interface-qualified-name-binds-its-own-slot + ;; The form defrecord expands to. + (is (= [2 1] (m/eval '(do (defrecord Pair [a b]) + (let [p (->Pair 1 2)] + [(count p) (:a p)])))))) + +(deftest overloads-outside-clojure-lang + ;; IDictionary.GetEnumerator and IEnumerable.GetEnumerator are two slots; the + ;; BCL collection interfaces carry more of these than clojure.lang does. + ;; Two written overloads stay two methods on the emitted type; the merging + ;; that count gets must not reach a name whose return types differ. + (is (= [2 true 2] + (m/eval '(do (deftype Dict [m] + System.Collections.IDictionary + (^System.Collections.IDictionaryEnumerator GetEnumerator [_] nil) + (^System.Collections.IEnumerator GetEnumerator [_] nil) + (get_Count [_] (count m)) + (Contains [_ k] (contains? m k))) + (let [d (Dict. {:a 1 :b 2})] + [(.Count ^System.Collections.ICollection d) + (.Contains d :a) + (->> (.GetMethods Dict (enum-or System.Reflection.BindingFlags/Instance + System.Reflection.BindingFlags/Public + System.Reflection.BindingFlags/DeclaredOnly)) + (filter #(= "GetEnumerator" (.Name %))) + count)])))))) + +(deftest reify-covers-the-slot-too + (is (= 2 (m/eval '(count (reify clojure.lang.IPersistentMap + (count [_] 2) + (seq [_] nil))))))) + +(deftest proxy-covers-the-slot-too + ;; proxy binds its methods through the same pass but omits this from the + ;; params, and its defaults are split across a super map and an interface one. + (is (= [7 7] (m/eval '(let [p (proxy [clojure.lang.IPersistentMap] [] + (count [] 7) + (seq [] nil))] + [(count p) (.count ^clojure.lang.Counted p)]))))) + +(deftest unknown-method-reports-a-plain-miss + ;; A name no interface declares is not an overload, so it keeps the generic + ;; message rather than the return type hint one. + (is (thrown-with-msg? + clojure.lang.ClojureException #"No match binding method" + (m/eval '(deftype Absent [] + clojure.lang.Counted + (nonexistent [_] 1)))))) diff --git a/magic-compiler/test/magic/test/errors.clj b/magic-compiler/test/magic/test/errors.clj index 7251e325d..b31566fbc 100644 --- a/magic-compiler/test/magic/test/errors.clj +++ b/magic-compiler/test/magic/test/errors.clj @@ -113,3 +113,25 @@ (str (m/eval (list 'fn [] (System.Random.)))) (catch Exception e (root-message e)))] (is (string/includes? msg "print-dup"))))) + +;;; a non-IFn callee throws a catchable cast error rather than failing verification + +(defn- root-type-name [^Exception e] + (if-let [inner (.InnerException e)] + (recur inner) + (.Name (.GetType e)))) + +(defn- eval-outcome [form] + (try (m/eval form) :no-throw + (catch Exception e (root-type-name e)))) + +(deftest non-ifn-callee-throws-cast-error + (testing "a value-typed callee is boxed, so the method verifies and the cast throws" + (doseq [form ['(1 2) ; boxed + '(true 2) ; a const Boolean converts to Magic.Constants instead + '(let [x (int 1)] (x 2))]] ; value-typed, and not a literal + (is (= "InvalidCastException" (eval-outcome form)) (pr-str form)))) + (testing "a reference-typed callee already behaved" + (is (= "InvalidCastException" (eval-outcome '("s" 2))))) + (testing "an IFn callee still runs" + (is (= 20 (m/eval '([10 20] 1)))))) diff --git a/magic-compiler/test/magic/test/mage.clj b/magic-compiler/test/magic/test/mage.clj new file mode 100644 index 000000000..38149654b --- /dev/null +++ b/magic-compiler/test/magic/test/mage.clj @@ -0,0 +1,15 @@ +(ns magic.test.mage + (:require [mage.core :as il] + clojure.test)) + +;;; every short arity of il/type reaches the full one + +(def ^:private full + (il/type "T" System.Reflection.TypeAttributes/Public [] System.Object nil [] [])) + +(clojure.test/deftest test-type-short-arities + (clojure.test/is (= full (il/type "T"))) + (clojure.test/is (= full (il/type "T" []))) + (clojure.test/is (= full (il/type "T" [] []))) + (clojure.test/is (= full (il/type "T" System.Reflection.TypeAttributes/Public [] []))) + (clojure.test/is (= full (il/type "T" System.Reflection.TypeAttributes/Public [] System.Object [])))) diff --git a/magic-compiler/test/magic/test/numbers.clj b/magic-compiler/test/magic/test/numbers.clj index f7fb23b3b..fa862284e 100644 --- a/magic-compiler/test/magic/test/numbers.clj +++ b/magic-compiler/test/magic/test/numbers.clj @@ -1,5 +1,6 @@ (ns magic.test.numbers - (:require [clojure.test :refer [deftest testing]]) + (:require [clojure.test :refer [deftest testing]] + [magic.api :as m]) (:use magic.test.common)) (deftest unchecked-cast-char @@ -16,6 +17,106 @@ (cljclr=magic (char (rand-nth [65]))) (cljclr=magic (int (rand-nth [7])))) +(deftest boxed-ulong-cast + (clojure.test/is (true? (m/eval '(== 1 (long (identity (ulong 1))))))) + (clojure.test/is (true? (m/eval '(== 1 (int (identity (ulong 1))))))) + (clojure.test/is (= :threw (m/eval '(try (long (identity (ulong 18446744073709551615))) + (catch ArgumentException e :threw)))))) + +(def cast-targets '[byte sbyte short ushort int uint long ulong char]) +(def cast-sources '[byte sbyte short ushort int uint long ulong char float double]) + +(defn cast-form + "Compares inside the compiled form, because an unsigned result does not + survive the host boundary." + [target source v] + (read-string + (str "(try (== " v " ((fn [^" source " x] (" target " x)) " v "))" + " (catch ArgumentException e :threw))"))) + +(deftest checked-narrowing-every-source-target-pair + (testing "a value every type can hold converts, so no pair loses its conversion" + (doseq [target cast-targets + source cast-sources] + (clojure.test/is (true? (m/eval (cast-form target source 1))) + (str "(" target " ^" source " 1)")))) + (testing "a value the target cannot hold throws, and only where it cannot hold it" + (doseq [[target source v] [['byte 'long 300] + ['byte 'int 300] + ['byte 'ushort 300] + ['sbyte 'byte 200] + ['short 'uint 70000] + ['short 'char 65535] + ['ushort 'long 70000] + ['ushort 'int -1] + ['int 'long 4294967296] + ['int 'ulong 18446744073709551615] + ['int 'double 1e300] + ['int 'float 1e30] + ['uint 'long -1] + ['uint 'double 1e300] + ['long 'ulong 18446744073709551615] + ['long 'double 1e300] + ['ulong 'long -1] + ['ulong 'int -1] + ['char 'long 100000] + ['char 'int -1] + ['char 'ulong 100000]]] + (clojure.test/is (= :threw (m/eval (cast-form target source v))) + (str "(" target " ^" source " " v ")"))))) + +(deftest literal-narrowing-matches-runtime-semantics + (testing "an out-of-range literal throws at runtime, catchable, instead of aborting compilation" + (clojure.test/are [form] (= :threw (m/eval form)) + '(try (int 4294967296) (catch ArgumentException e :threw)) + '(try (uint -1) (catch ArgumentException e :threw)) + '(try (byte 300) (catch ArgumentException e :threw)) + '(try (ulong -1) (catch ArgumentException e :threw)))) + (testing "a fractional literal truncates as the runtime cast does, not Convert's rounding" + ;; explicit expected values: cljclr=magic compares two MAGIC compilation + ;; paths and passes when both share the bug + (clojure.test/is (== 1 (m/eval '(int 1.5)))) + (clojure.test/is (== 2 (m/eval '(int 2.5)))) + (clojure.test/is (== -1 (m/eval '(long -1.5)))) + (clojure.test/is (== 1 (m/eval '(byte 1.5))))) + (testing "an in-range literal still reinterprets" + (clojure.test/is (== 42 (m/eval '(int 42.0)))) + (clojure.test/is (== 1.5 (m/eval '(float 1.5)))) + (clojure.test/is (true? (m/eval '(== 4294967295 (uint 4294967295))))))) + +(deftest checked-narrowing-keeps-value-preserving-conversions + (cljclr=magic ((fn [^int x] (long x)) 7)) + (cljclr=magic ((fn [^uint x] (long x)) (uint 4294967295))) + (cljclr=magic ((fn [^uint x] (ulong x)) (uint 4294967295))) + (cljclr=magic ((fn [^char x] (int x)) \uFFFF)) + (cljclr=magic ((fn [^long x] (int x)) 7)) + (cljclr=magic ((fn [^double x] (int x)) Double/NaN))) + +(deftest unchecked-narrowing-cast-still-wraps + (cljclr=magic ((fn [^long x] (unchecked-int x)) 4294967296)) + (clojure.test/is + (zero? (binding [*unchecked-math* true] + (m/eval '((fn [^long x] (int x)) 4294967296)))))) + +(deftest checked-narrowing-call-sites + (clojure.test/are [form] (= :threw (m/eval form)) + '(try (let [a (int-array [10 20 30])] ((fn [^long i] (aget a i)) 4294967296)) + (catch ArgumentException e :threw)) + '(try (let [a (int-array [10 20 30])] ((fn [^long i] (nth a i)) 4294967296)) + (catch ArgumentException e :threw)) + '(try (let [a (int-array [10 20 30])] ((fn [^long i] (aset a i 99)) 4294967296)) + (catch ArgumentException e :threw)) + '(try (let [a (int-array [10 20 30])] ((fn [^long v] (aset a 0 v)) 4294967296)) + (catch ArgumentException e :threw)) + '(try ((fn [^long n] (make-array Int32 n)) 4294967296) + (catch ArgumentException e :threw)) + '(try ((fn [^long i] (.Substring "hello" i)) 4294967296) + (catch ArgumentException e :threw)) + '(try ((fn [^long i] (System.Char/ConvertFromUtf32 i)) 4294967296) + (catch ArgumentException e :threw)) + '(try ((fn [^long n] (String. \a n)) 4294967296) + (catch ArgumentException e :threw)))) + (deftest promote-narrow-integer-arithmetic (cljclr=magic (inc UInt32/MaxValue)) (cljclr=magic (inc UInt16/MaxValue)) diff --git a/magic-compiler/test/magic/test/stdlib.clj b/magic-compiler/test/magic/test/stdlib.clj index ca078e8c7..d87f0219e 100644 --- a/magic-compiler/test/magic/test/stdlib.clj +++ b/magic-compiler/test/magic/test/stdlib.clj @@ -226,3 +226,98 @@ (= :from-meta (clojure.core.protocols/datafy (with-meta {} {`clojure.core.protocols/datafy (fn [_] :from-meta)}))))) + +;;; spit: a write truncates, :append appends (JVM-verified expected values) + +(defn- with-spit-file [f] + (let [path (System.IO.Path/Combine (System.IO.Path/GetTempPath) + (str (gensym "magic-spit-test") ".txt"))] + (try (f path) + (finally (System.IO.File/Delete path))))) + +(deftest test-spit-truncates + (with-spit-file + (fn [path] + (spit path "DATA-PRESENT") + (spit path "AB") + (clojure.test/is (= "AB" (slurp path)))))) + +(deftest test-spit-empty-clears + (with-spit-file + (fn [path] + (spit path "DATA-PRESENT") + (spit path nil) + (clojure.test/is (= "" (slurp path)))))) + +(deftest test-spit-append + (with-spit-file + (fn [path] + (spit path "AAA") + (spit path "BBB" :append true) + (clojure.test/is (= "AAABBB" (slurp path)))))) + +(deftest test-spit-file-mode-still-wins + (with-spit-file + (fn [path] + (spit path "AAA") + (spit path "BBB" :file-mode System.IO.FileMode/Append) + (clojure.test/is (= "AAABBB" (slurp path)))))) + +;;; #object[...] carries the qualified type name, as (.getName c) does on the JVM + +(defn- object-tag [o] + (second (re-find #"^#object\[(\S+) " (pr-str o)))) + +(deftest test-print-tagged-object-qualified-name + (clojure.test/is (= "System.Text.StringBuilder" (object-tag (System.Text.StringBuilder.)))) + (clojure.test/is (= "clojure.lang.Atom" (object-tag (atom 1))))) + +;;; #error carries the :message value, not just the label + +(deftest test-print-throwable-message + (clojure.test/is (= "\"boom\"" + (second (re-find #"\n :message (\S+)\n" + (pr-str (ex-info "boom" {:a 1}))))))) + +;;; defn records a qualified arglist :tag, as (.getName c) does on the JVM + +(defn tagged-fn ^StringBuilder [] (System.Text.StringBuilder.)) + +(deftest test-defn-arglist-tag-qualified + (clojure.test/is (= 'System.Text.StringBuilder + (:tag (meta (first (:arglists (meta #'tagged-fn)))))))) + +;;; sort carries the collection's metadata (1.10, CLJ-2417) + +(deftest test-sort-retains-meta + (clojure.test/is (= {:x 1} (meta (sort (with-meta [3 1 2] {:x 1}))))) + (clojure.test/is (= [1 2 3] (sort [3 1 2])))) + +;;; namespace maps print in the map's own key order (1.10, CLJ-2469) + +(deftest test-namespace-map-key-order + (binding [*print-namespace-maps* true] + (clojure.test/is (= "#:a{:k0 0, :k1 1, :k2 2, :k3 3, :k4 4, :k5 5, :k6 6, :k7 7, :k8 8, :k9 9}" + (pr-str (array-map :a/k0 0 :a/k1 1 :a/k2 2 :a/k3 3 :a/k4 4 + :a/k5 5 :a/k6 6 :a/k7 7 :a/k8 8 :a/k9 9)))) + (clojure.test/is (= "{:x 1, :a/y 2}" + (pr-str (array-map :x 1 :a/y 2))) + "an unqualified key blocks the lift"))) + +;;; pprint writes collection metadata under *print-meta* (1.10, CLJ-1445) + +(deftest test-pprint-print-meta + (require 'clojure.pprint) + (let [pp (resolve 'clojure.pprint/pprint)] + (binding [*print-meta* true] + (clojure.test/is (= "^{:x 1} [1 2]\n" (with-out-str (pp (with-meta [1 2] {:x 1}))))) + (clojure.test/is (= "^{:x 1} #{1}\n" (with-out-str (pp (with-meta #{1} {:x 1})))))) + (clojure.test/is (= "[1 2]\n" (with-out-str (pp (with-meta [1 2] {:x 1})))) + "no metadata written when *print-meta* is false"))) + +;;; doc prints a special form's docstring once (1.10, CLJ-2295) + +(deftest test-doc-special-form-once + (require 'clojure.repl) + (let [out (with-out-str (eval '(clojure.repl/doc if)))] + (clojure.test/is (= 1 (count (re-seq #"Evaluates test" out)))))) diff --git a/magic-compiler/test/magic/test/string.clj b/magic-compiler/test/magic/test/string.clj index 624593522..3ef70ad87 100644 --- a/magic-compiler/test/magic/test/string.clj +++ b/magic-compiler/test/magic/test/string.clj @@ -5,7 +5,20 @@ (deftest t-split (is (= ["a" "b"] (clojure.string/split "a-b" #"-"))) (is (= ["a" "b-c"] (clojure.string/split "a-b-c" #"-" 2))) - (is (vector? (clojure.string/split "abc" #"-")))) + (is (vector? (clojure.string/split "abc" #"-"))) + (is (= ["a" "b"] (clojure.string/split "a b " #" "))) + (is (= ["a"] (clojure.string/split "a,," #","))) + (is (= [] (clojure.string/split "," #","))) + (is (= [""] (clojure.string/split "" #","))) + (is (= ["a" "" "b"] (clojure.string/split "a,,b" #","))) + (is (= ["a" "b"] (clojure.string/split "a b " #" " 0))) + (is (= ["a" "b" ""] (clojure.string/split "a b " #" " -1))) + (is (= ["a" "b" "c"] (clojure.string/split "abc" #""))) + (is (= ["a" "b" "c" ""] (clojure.string/split "abc" #"" -1))) + (is (= [""] (clojure.string/split "" #""))) + (is (= ["" "a"] (clojure.string/split ",a" #","))) + (is (= ["" "" "a"] (clojure.string/split "XaX" #"X*"))) + (is (= ["a" "" "b" "" "c"] (clojure.string/split "aXbXc" #"X*")))) (deftest t-reverse (is (= "tab" (clojure.string/reverse "bat")))) @@ -103,7 +116,14 @@ (deftest t-split-lines (is (= ["one" "two" "three"] (clojure.string/split-lines "one\ntwo\r\nthree"))) (is (vector? (clojure.string/split-lines "one\ntwo\r\nthree"))) - (is (= (list "foo") (clojure.string/split-lines "foo")))) + (is (= (list "foo") (clojure.string/split-lines "foo"))) + (is (= ["a" "b"] (clojure.string/split-lines "a\nb\n"))) + (is (= ["a" "b"] (clojure.string/split-lines "a\r\nb\r\n"))) + (is (= [""] (clojure.string/split-lines ""))) + (is (= [] (clojure.string/split-lines "\n"))) + (is (= [] (clojure.string/split-lines "\n\n"))) + (is (= ["foo"] (clojure.string/split-lines "foo\n\n"))) + (is (= ["" "bar"] (clojure.string/split-lines "\nbar")))) (deftest t-index-of (is (let [sb "tacos"] (= 2 (clojure.string/index-of sb "c")))) diff --git a/magic-unity/Editor/Reload.meta b/magic-unity/Editor/Reload.meta new file mode 100644 index 000000000..f417320be --- /dev/null +++ b/magic-unity/Editor/Reload.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 5795a4ce8b1654bffb769e082e84888c +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/magic-unity/Editor/Reload/ClojureReloader.cs b/magic-unity/Editor/Reload/ClojureReloader.cs new file mode 100644 index 000000000..69b53fec5 --- /dev/null +++ b/magic-unity/Editor/Reload/ClojureReloader.cs @@ -0,0 +1,221 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.IO; +using System.Linq; + +namespace Magic.Unity +{ + /// + /// Effective only with a runtime that loads the code from the source + /// (ClojureCLR). + /// + /// A file event marks a path as dirty on the background watcher thread. One + /// single thread calls , which reads the file, evaluates it, + /// and retries a read that fails. + /// + /// The roots are captured at construction: to watch another set, dispose + /// this one and build a new one. + /// + public sealed class ClojureReloader : IDisposable + { + struct Retry + { + public int Attempts; + public long DueMs; + } + + const int MaxReadAttempts = 2; + + const long RetryBackoffMs = 150; + + readonly DebouncedFileWatcher _watcher; + readonly Action _logger; + readonly Action _onChanged; + bool _disposed; + + readonly Dictionary _retries = new Dictionary(StringComparer.Ordinal); + + readonly Stopwatch _clock = Stopwatch.StartNew(); + + /// + /// Monitors the source roots. Logs if no root is given, or if a root is + /// absent. Only a null throws. + /// + /// The reporting mechanism. If null, exceptions are + /// skipped too. This callback must not throw an exception. + /// A synchronous callback after each reload. + public ClojureReloader(IEnumerable roots, Action logger, Action onChanged = null) + { + if (roots == null) + { + throw new ArgumentNullException(nameof(roots)); + } + _logger = logger; + _onChanged = onChanged; + var captured = roots.ToArray(); + if (captured.Length == 0) + { + _logger?.Invoke("No clj source root. Reload disabled."); + return; + } + _watcher = new DebouncedFileWatcher(IsClojureSource, _logger); + foreach (var root in captured) + { + bool watched; + try + { + watched = _watcher.AddRoot(root); + } + catch (Exception ex) + { + _logger?.Invoke($"Failed to watch clj source root, skipped. {root}: {ex}"); + continue; + } + if (watched) + { + _logger?.Invoke($"Watching clj source root. {root}"); + } + } + } + + /// + /// Call this method on each frame or tick of the main thread of the host. + /// It evaluates each stable file change, then the read retries that are due. + /// Does nothing when no root is watched or after Dispose. Handles + /// IOException or UnauthorizedAccessException by retrying. + /// Do not call `Poll` from the onChanged callback. + /// + public void Poll() + { + if (_disposed || _watcher == null) // a disposed watcher still hands out pending marks + { + return; + } + foreach (var path in _watcher.TakeSettled()) + { + _retries.Remove(path); // otherwise, new content keeps the failure count of previous content + Attempt(path); + } + foreach (var path in DueRetries()) + { + if (_watcher.IsPending(path)) + { + _retries.Remove(path); + _logger?.Invoke($"Reload retry for {path} superseded by a pending file event."); + continue; + } + Attempt(path); + } + } + + /// + /// Idempotent, and callable from the onChanged callback. + /// + public void Dispose() + { + _disposed = true; + _watcher?.Dispose(); + _retries.Clear(); + } + + void Attempt(string path) + { + // Check here, since Dispose() can be called in the onChanged callback + if (_disposed) + { + return; + } + // Compiler.load evaluates one form after the other, thus a save during + // the evaluation would give mixed content. + string snapshot; + try + { + snapshot = File.ReadAllText(path); + } + catch (Exception ex) when (ex is FileNotFoundException || ex is DirectoryNotFoundException) + { + // These errors can be thrown if a file was deleted, and are inherited from IOException + _retries.Remove(path); + return; + } + catch (Exception ex) when (ex is IOException || ex is UnauthorizedAccessException) + { + // On Windows a rename-replace operation throws + // UnauthorizedAccessException for a short time, and not IOException. + Defer(path, ex); + return; + } + // The budget is only for the conflicts between a read and a save: code + // that does not compile stays the same after each new read. + _retries.Remove(path); + try + { + clojure.lang.Compiler.load(new StringReader(snapshot), + path, Path.GetFileName(path), path); // the reader conditionals need the name of the .cljc file + } + catch (Exception ex) + { + _logger?.Invoke($"Reload failed for {path}: {ex}"); + return; + } + _logger?.Invoke($"Reloaded {path}"); + try + { + _onChanged?.Invoke(path); + } + catch (Exception ex) + { + _logger?.Invoke($"Reload callback failed for {path}: {ex}"); + } + } + + void Defer(string path, Exception ex) + { + _retries.TryGetValue(path, out var retry); + retry.Attempts++; + if (retry.Attempts >= MaxReadAttempts) + { + _retries.Remove(path); + _logger?.Invoke( + $"Reload failed for {path} (unreadable after {MaxReadAttempts} attempts): {ex.Message}" + ); + return; + } + retry.DueMs = _clock.ElapsedMilliseconds + RetryBackoffMs; + _retries[path] = retry; + _logger?.Invoke( + $"Reload deferred (attempt {retry.Attempts}/{MaxReadAttempts}) for {path}: {ex.Message}" + ); + } + + List DueRetries() + { + var due = new List(); + var now = _clock.ElapsedMilliseconds; + foreach (var kv in _retries) + { + if (now >= kv.Value.DueMs) + { + due.Add(kv.Key); + } + } + return due; + } + + static bool IsClojureSource(string path) + { + var name = Path.GetFileName(path); + // Prefix checks - Emacs names its lock file `.#core.clj` + if (name.Length == 0 || name[0] == '.' || name[0] == '#') + { + return false; + } + // Case-insensitive suffix checks + var ext = Path.GetExtension(name); + return ext.Equals(".clj", StringComparison.OrdinalIgnoreCase) + || ext.Equals(".cljc", StringComparison.OrdinalIgnoreCase) + || ext.Equals(".cljr", StringComparison.OrdinalIgnoreCase); + } + } +} diff --git a/magic-unity/Editor/Reload/ClojureReloader.cs.meta b/magic-unity/Editor/Reload/ClojureReloader.cs.meta new file mode 100644 index 000000000..33251a38c --- /dev/null +++ b/magic-unity/Editor/Reload/ClojureReloader.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: 712186df1ffb54ac39b08b9834cf5ce7 +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/magic-unity/Editor/Reload/DebouncedFileWatcher.cs b/magic-unity/Editor/Reload/DebouncedFileWatcher.cs new file mode 100644 index 000000000..7ccf26137 --- /dev/null +++ b/magic-unity/Editor/Reload/DebouncedFileWatcher.cs @@ -0,0 +1,266 @@ +using System; +using System.Collections.Concurrent; +using System.Collections.Generic; +using System.Diagnostics; +using System.IO; + +namespace Magic.Unity +{ + /// + /// Turns the event stream into a debounced list + /// of changed paths, drained on the caller's thread via . + /// One editor save commonly fires 2-3 FSW events (content + metadata writes) on + /// arbitrary ThreadPool threads. + /// + /// Watcher threads produce, a single thread consumes. + /// + /// A watcher is always removed from the table before it is disposed: disposal + /// itself trips Error, and a watcher still in the table would be re-armed. + /// + internal sealed class DebouncedFileWatcher : IDisposable + { + const int InternalBufferBytes = 64 * 1024; + + const int SettleMs = 200; + + readonly Func _accept; + readonly Action _log; + + readonly ConcurrentDictionary _pending = + new ConcurrentDictionary(StringComparer.Ordinal); + + readonly Dictionary _watchersByRoot = + new Dictionary(StringComparer.Ordinal); + readonly object _watchersLock = new object(); + + readonly Stopwatch _clock = Stopwatch.StartNew(); + + volatile bool _disposed; + + /// Must not be null. + /// May be null. Must not throw -- an exception escaping an FSW + /// handler is process-fatal. + public DebouncedFileWatcher(Func accept, Action log) + { + if (accept == null) + { + throw new ArgumentNullException(nameof(accept)); + } + _accept = accept; + _log = log; + } + + /// + /// Watch recursively. Collapses existing overlapping roots. + /// + /// True if newly armed or already watched. + public bool AddRoot(string dir) + { + if (string.IsNullOrEmpty(dir)) + { + return false; + } + var full = Normalize(dir); + lock (_watchersLock) + { + if (_disposed) + { + throw new ObjectDisposedException(nameof(DebouncedFileWatcher)); + } + if (!Directory.Exists(full)) + { + _log?.Invoke($"DebouncedFileWatcher: no such directory, not watching. {full}"); + return false; + } + var covered = new List(); + foreach (var existing in _watchersByRoot.Keys) + { + if (Covers(existing, full)) + { + return true; + } + if (Covers(full, existing)) + { + covered.Add(existing); + } + } + foreach (var c in covered) + { + var dead = _watchersByRoot[c]; + _watchersByRoot.Remove(c); + dead.Dispose(); + } + _watchersByRoot[full] = Arm(full); + return true; + } + } + + FileSystemWatcher Arm(string dir) + { + var w = new FileSystemWatcher(dir) + { + // FileName is for the editors that save by writing a temp file and + // renaming it over the target (Vim, Emacs) + NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.FileName, + // Filter isn't passed to the OS -- all events are buffered anyway. So just + // filter with _accept. + Filter = "*", + IncludeSubdirectories = true, + // Windows: a burst (git checkout, format-on-save) overflows the buffer + // and drops events. + InternalBufferSize = InternalBufferBytes, + }; + w.Changed += OnFsEvent; + w.Created += OnFsEvent; + w.Renamed += OnFsEvent; // for Emacs, a backup rename to `foo.clj~` should be rejected by `_accept` + w.Error += OnError; + w.EnableRaisingEvents = true; + return w; + } + + void OnFsEvent(object sender, FileSystemEventArgs e) + { + Mark(e.FullPath); + } + + void Mark(string path) + { + if (_disposed || !_accept(path)) + { + return; + } + _pending[path] = _clock.ElapsedMilliseconds; // last write wins + } + + void OnError(object sender, ErrorEventArgs e) + { + // Raised when the internal buffer overflowed (a burst dropped events) or the + // watch broke. The dropped changes are lost; a subsequent save re-marks them. + var w = sender as FileSystemWatcher; + if (_disposed || w == null) + { + return; + } + lock (_watchersLock) + { + if (RootOf(w) == null) + { + return; + } + } + _log?.Invoke($"DebouncedFileWatcher error, re-arming: {e.GetException()?.Message}"); + try + { + // The toggle follows FSW's own Restart(), see + // https://github.com/microsoft/referencesource/blob/main/System/services/io/system/io/FileSystemWatcher.cs + w.EnableRaisingEvents = false; + w.EnableRaisingEvents = true; + } + catch (Exception ex) + { + _log?.Invoke( + $"DebouncedFileWatcher re-arm failed for {w.Path}, no longer watching: {ex.Message}" + ); + lock (_watchersLock) + { + var key = RootOf(w); + if (key != null) + { + _watchersByRoot.Remove(key); + } + } + w.Dispose(); + } + } + + // Finds the table key by identity: w.Path may not echo the Normalize()d key verbatim. + string RootOf(FileSystemWatcher w) + { + foreach (var kv in _watchersByRoot) + { + if (ReferenceEquals(kv.Value, w)) + { + return kv.Key; + } + } + return null; + } + + /// + /// Stop tracking and return paths that have been quiet for the settle window. + /// + /// Order is unspecified: timestamps are last-touch, not save order. + /// + public List TakeSettled() + { + var now = _clock.ElapsedMilliseconds; + var result = new List(); + foreach (var kv in _pending) + { + if (now - kv.Value < SettleMs) + { + continue; + } + // Compare-and-remove: a watcher thread may have re-marked this path + if (RemoveIf(kv.Key, kv.Value)) + { + result.Add(kv.Key); + } + } + return result; + } + + /// + /// True while a mark is held that a future will + /// return, settled or not. + /// + public bool IsPending(string path) + { + return _pending.ContainsKey(path); + } + + // The TryRemove(key, value, out) to compare before removing is .NET 5+; + // this is .NET Framework 4.x-compatible. + bool RemoveIf(string path, long expected) + { + return ((ICollection>)_pending).Remove( + new KeyValuePair(path, expected) + ); + } + + static string Normalize(string dir) + { + var full = Path.GetFullPath(dir); + return full.TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar); + } + + // Segment-aware: foo/bar covers foo/bar/baz, never foo/barbaz. + static bool Covers(string ancestor, string descendant) + { + if (string.Equals(ancestor, descendant, StringComparison.Ordinal)) + { + return true; + } + return descendant.StartsWith(ancestor + Path.DirectorySeparatorChar, StringComparison.Ordinal); + } + + // Does not wait for a handler already running on a ThreadPool thread, so + // events can still arrive after this returns. + public void Dispose() + { + lock (_watchersLock) + { + if (_disposed) + { + return; + } + _disposed = true; + foreach (var w in _watchersByRoot.Values) + { + w.Dispose(); + } + _watchersByRoot.Clear(); + } + } + } +} diff --git a/magic-unity/Editor/Reload/DebouncedFileWatcher.cs.meta b/magic-unity/Editor/Reload/DebouncedFileWatcher.cs.meta new file mode 100644 index 000000000..9c2b27f4d --- /dev/null +++ b/magic-unity/Editor/Reload/DebouncedFileWatcher.cs.meta @@ -0,0 +1,11 @@ +fileFormatVersion: 2 +guid: eb778222030fa4f88a1ccdccf88d1f21 +MonoImporter: + externalObjects: {} + serializedVersion: 2 + defaultReferences: [] + executionOrder: 0 + icon: {instanceID: 0} + userData: + assetBundleName: + assetBundleVariant: diff --git a/magic-unity/Editor/Reload/Magic.Unity.Editor.Reload.asmdef b/magic-unity/Editor/Reload/Magic.Unity.Editor.Reload.asmdef new file mode 100644 index 000000000..e476ab1d1 --- /dev/null +++ b/magic-unity/Editor/Reload/Magic.Unity.Editor.Reload.asmdef @@ -0,0 +1,20 @@ +{ + "name": "Magic.Unity.Editor.Reload", + "rootNamespace": "", + "references": [], + "includePlatforms": [ + "Editor" + ], + "excludePlatforms": [], + "allowUnsafeCode": false, + "overrideReferences": true, + "precompiledReferences": [ + "Clojure.dll" + ], + "autoReferenced": true, + "defineConstraints": [ + "!MAGIC_RUNTIME_IN_EDITOR" + ], + "versionDefines": [], + "noEngineReferences": true +} diff --git a/magic-unity/Editor/Reload/Magic.Unity.Editor.Reload.asmdef.meta b/magic-unity/Editor/Reload/Magic.Unity.Editor.Reload.asmdef.meta new file mode 100644 index 000000000..2e63c872c --- /dev/null +++ b/magic-unity/Editor/Reload/Magic.Unity.Editor.Reload.asmdef.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: 7b3bf7114d6bd4f778ceee7f5aa97f1b +AssemblyDefinitionImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/magic-unity/README.md b/magic-unity/README.md index 7fe425b50..461d2762f 100644 --- a/magic-unity/README.md +++ b/magic-unity/README.md @@ -2,11 +2,9 @@ [Unity](https://unity.com/) integration for the MAGIC compiler. -This file is the developer reference. For the full usage guide about using with Unity, see the [Unity integration guide](https://github.com/flybot-sg/magic/blob/main/docs/unity-integration.md). +This UPM package lets a Unity game run Clojure, in the Editor and in a shipped player on every backend Unity supports, IL2CPP included (iOS, Android, consoles). It ships two Clojure runtimes (MAGIC and ClojureCLR), a small C# API for calling into Clojure, and the Editor build hooks that make MAGIC's IL survive AOT compilation. It does not compile Clojure: namespaces are compiled to plugin DLLs outside Unity with `nos build`, and Unity loads them as plain .NET assemblies. -This UPM package lets a Unity game run Clojure, in the Editor and in a shipped player on every backend Unity supports, IL2CPP included (iOS, Android, consoles). It ships two Clojure runtimes (MAGIC and ClojureCLR), a small C# API for calling into Clojure, and the Editor build hooks that make MAGIC's IL survive AOT compilation. - -It does not compile Clojure. Namespaces are compiled to plugin DLLs outside Unity with `nos build`, and Unity loads them as plain .NET assemblies; see [step 4 of the guide](https://github.com/flybot-sg/magic/blob/main/docs/unity-integration.md#steps). +This file is the API reference. The workflow — project setup, `nos build`, choosing the Editor runtime, IL2CPP — is in the [Unity integration guide](https://github.com/flybot-sg/magic/blob/main/docs/unity-integration.md). ## Install @@ -18,15 +16,13 @@ Add to `Packages/manifest.json`, pinned to a tag from the [releases page](https: ## Runtime API -`Magic.Unity.Clojure` static class, available on all platforms: +`Magic.Unity.Clojure` static class, available on all platforms. Nothing calls these for you; a `MonoBehaviour` of yours has to. See [step 5 of the guide](https://github.com/flybot-sg/magic/blob/main/docs/unity-integration.md#steps). - `void Require(string ns)` - load a Clojure namespace. Must be called before looking up vars in that namespace. - `clojure.lang.Var GetVar(string ns, string name)` - look up a Clojure var. Dereference with `deref` or invoke with `invoke`. - `T GetVar(string ns, string name)` - typed variant. - `void Boot()` - initialize the Clojure runtime. Called automatically by the other methods; rarely needed directly. -Nothing calls these for you; a `MonoBehaviour` of yours has to. See [step 5 of the guide](https://github.com/flybot-sg/magic/blob/main/docs/unity-integration.md#steps) for the pattern. - ## Editor API `Magic.Unity.EditorRuntime` static class, Editor-only: @@ -34,6 +30,16 @@ Nothing calls these for you; a `MonoBehaviour` of yours has to. See [step 5 of t - `bool IsMagicEnabled()` - whether the Editor's Clojure runtime is MAGIC. - `void UseMagic()` / `void UseClojureCLR()` - set it, on the active build target. It triggers a recompilation, so the switch takes effect on the next Unity invocation. +`Magic.Unity.ClojureReloader` class, in the Editor-only assembly `Magic.Unity.Editor.Reload`, which exists only while the Editor runs ClojureCLR (`MAGIC_RUNTIME_IN_EDITOR` unset). Wrap code that uses it in `#if UNITY_EDITOR && !MAGIC_RUNTIME_IN_EDITOR`. + +- `ClojureReloader(IEnumerable roots, Action logger, Action onChanged = null)` - watch the given directories recursively for `.clj`/`.cljc`/`.cljr` saves; typically the directories you put on `CLOJURE_LOAD_PATH`. +- `void Poll()` - call it on each tick of one thread, normally the main thread. It evaluates each source file that has settled. +- `void Dispose()` - stop watching. + +Files saved together reload in no particular order; there is no dependency ordering between namespaces. A file that loads before its dependency fails and is not retried, so needs to be saved again. + +The XML doc comments on the class carry the per-method contract. + ## Examples [magic-unity-smoke](https://github.com/flybot-sg/magic/tree/main/unity-examples/magic-unity-smoke) is a working IL2CPP regression project built on this package; [magic-unity-coexist](https://github.com/flybot-sg/magic/tree/main/unity-examples/magic-unity-coexist) is the headless regression for both Editor-runtime states. diff --git a/magic-unity/Runtime/clojure-clr/Clojure.Source.dll b/magic-unity/Runtime/clojure-clr/Clojure.Source.dll index fa91b6e88..758eb7ce8 100644 Binary files a/magic-unity/Runtime/clojure-clr/Clojure.Source.dll and b/magic-unity/Runtime/clojure-clr/Clojure.Source.dll differ diff --git a/magic-unity/Runtime/clojure-clr/Clojure.dll b/magic-unity/Runtime/clojure-clr/Clojure.dll index d8d26c3c7..8f8ea136b 100644 Binary files a/magic-unity/Runtime/clojure-clr/Clojure.dll and b/magic-unity/Runtime/clojure-clr/Clojure.dll differ diff --git a/magic-unity/Runtime/magic/Clojure.dll b/magic-unity/Runtime/magic/Clojure.dll index 876809581..a49c81029 100644 Binary files a/magic-unity/Runtime/magic/Clojure.dll and b/magic-unity/Runtime/magic/Clojure.dll differ diff --git a/magic-unity/Runtime/magic/clojure.clr.io.clj.dll b/magic-unity/Runtime/magic/clojure.clr.io.clj.dll index 38b618c91..70efbdd67 100755 Binary files a/magic-unity/Runtime/magic/clojure.clr.io.clj.dll and b/magic-unity/Runtime/magic/clojure.clr.io.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.clr.shell.clj.dll b/magic-unity/Runtime/magic/clojure.clr.shell.clj.dll index 14f3ebb66..1916b3f26 100755 Binary files a/magic-unity/Runtime/magic/clojure.clr.shell.clj.dll and b/magic-unity/Runtime/magic/clojure.clr.shell.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core.clj.dll b/magic-unity/Runtime/magic/clojure.core.clj.dll index d5b6d2f3e..d273c1819 100755 Binary files a/magic-unity/Runtime/magic/clojure.core.clj.dll and b/magic-unity/Runtime/magic/clojure.core.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core.protocols.clj.dll b/magic-unity/Runtime/magic/clojure.core.protocols.clj.dll index ec123614d..77b52751c 100755 Binary files a/magic-unity/Runtime/magic/clojure.core.protocols.clj.dll and b/magic-unity/Runtime/magic/clojure.core.protocols.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core.reducers.clj.dll b/magic-unity/Runtime/magic/clojure.core.reducers.clj.dll index 52241e7a9..354226dc1 100755 Binary files a/magic-unity/Runtime/magic/clojure.core.reducers.clj.dll and b/magic-unity/Runtime/magic/clojure.core.reducers.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core.server.clj.dll b/magic-unity/Runtime/magic/clojure.core.server.clj.dll index 56840382f..b96151394 100755 Binary files a/magic-unity/Runtime/magic/clojure.core.server.clj.dll and b/magic-unity/Runtime/magic/clojure.core.server.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core.specs.alpha.clj.dll b/magic-unity/Runtime/magic/clojure.core.specs.alpha.clj.dll index c04adee32..d6922127b 100755 Binary files a/magic-unity/Runtime/magic/clojure.core.specs.alpha.clj.dll and b/magic-unity/Runtime/magic/clojure.core.specs.alpha.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core_clr.clj.dll b/magic-unity/Runtime/magic/clojure.core_clr.clj.dll index 96d02d4e4..92cb5822d 100755 Binary files a/magic-unity/Runtime/magic/clojure.core_clr.clj.dll and b/magic-unity/Runtime/magic/clojure.core_clr.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core_deftype.clj.dll b/magic-unity/Runtime/magic/clojure.core_deftype.clj.dll index a7776b071..62b6e82d6 100755 Binary files a/magic-unity/Runtime/magic/clojure.core_deftype.clj.dll and b/magic-unity/Runtime/magic/clojure.core_deftype.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core_print.clj.dll b/magic-unity/Runtime/magic/clojure.core_print.clj.dll index e03552598..2ad29b2b8 100755 Binary files a/magic-unity/Runtime/magic/clojure.core_print.clj.dll and b/magic-unity/Runtime/magic/clojure.core_print.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.core_proxy.clj.dll b/magic-unity/Runtime/magic/clojure.core_proxy.clj.dll index ed046a63a..391f7e4bb 100755 Binary files a/magic-unity/Runtime/magic/clojure.core_proxy.clj.dll and b/magic-unity/Runtime/magic/clojure.core_proxy.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.data.clj.dll b/magic-unity/Runtime/magic/clojure.data.clj.dll index f7a4f5a90..eacf5684c 100755 Binary files a/magic-unity/Runtime/magic/clojure.data.clj.dll and b/magic-unity/Runtime/magic/clojure.data.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.instant.clj.dll b/magic-unity/Runtime/magic/clojure.instant.clj.dll index 0effa716a..87641a937 100755 Binary files a/magic-unity/Runtime/magic/clojure.instant.clj.dll and b/magic-unity/Runtime/magic/clojure.instant.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.main.clj.dll b/magic-unity/Runtime/magic/clojure.main.clj.dll index fb60098d6..2b0a4e3fe 100755 Binary files a/magic-unity/Runtime/magic/clojure.main.clj.dll and b/magic-unity/Runtime/magic/clojure.main.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.pprint.cl_format.clj.dll b/magic-unity/Runtime/magic/clojure.pprint.cl_format.clj.dll index 76f705cc0..b5919311a 100755 Binary files a/magic-unity/Runtime/magic/clojure.pprint.cl_format.clj.dll and b/magic-unity/Runtime/magic/clojure.pprint.cl_format.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.pprint.column_writer.clj.dll b/magic-unity/Runtime/magic/clojure.pprint.column_writer.clj.dll index 40216b156..32ac7091c 100755 Binary files a/magic-unity/Runtime/magic/clojure.pprint.column_writer.clj.dll and b/magic-unity/Runtime/magic/clojure.pprint.column_writer.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.pprint.dispatch.clj.dll b/magic-unity/Runtime/magic/clojure.pprint.dispatch.clj.dll index f7fea81c7..708aed855 100755 Binary files a/magic-unity/Runtime/magic/clojure.pprint.dispatch.clj.dll and b/magic-unity/Runtime/magic/clojure.pprint.dispatch.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.pprint.pprint_base.clj.dll b/magic-unity/Runtime/magic/clojure.pprint.pprint_base.clj.dll index de706ed98..2c3c1cb25 100755 Binary files a/magic-unity/Runtime/magic/clojure.pprint.pprint_base.clj.dll and b/magic-unity/Runtime/magic/clojure.pprint.pprint_base.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.pprint.pretty_writer.clj.dll b/magic-unity/Runtime/magic/clojure.pprint.pretty_writer.clj.dll index 662c180bc..6385f706d 100755 Binary files a/magic-unity/Runtime/magic/clojure.pprint.pretty_writer.clj.dll and b/magic-unity/Runtime/magic/clojure.pprint.pretty_writer.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.pprint.print_table.clj.dll b/magic-unity/Runtime/magic/clojure.pprint.print_table.clj.dll index b59845147..01fd2a9df 100755 Binary files a/magic-unity/Runtime/magic/clojure.pprint.print_table.clj.dll and b/magic-unity/Runtime/magic/clojure.pprint.print_table.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.pprint.utilities.clj.dll b/magic-unity/Runtime/magic/clojure.pprint.utilities.clj.dll index 29314a830..f9eee78ac 100755 Binary files a/magic-unity/Runtime/magic/clojure.pprint.utilities.clj.dll and b/magic-unity/Runtime/magic/clojure.pprint.utilities.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.repl.clj.dll b/magic-unity/Runtime/magic/clojure.repl.clj.dll index 6970b61c1..b935fee12 100755 Binary files a/magic-unity/Runtime/magic/clojure.repl.clj.dll and b/magic-unity/Runtime/magic/clojure.repl.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.set.clj.dll b/magic-unity/Runtime/magic/clojure.set.clj.dll index b5553f54f..3166d10d6 100755 Binary files a/magic-unity/Runtime/magic/clojure.set.clj.dll and b/magic-unity/Runtime/magic/clojure.set.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.spec.alpha.clj.dll b/magic-unity/Runtime/magic/clojure.spec.alpha.clj.dll index 3611c17d8..1e74276a7 100755 Binary files a/magic-unity/Runtime/magic/clojure.spec.alpha.clj.dll and b/magic-unity/Runtime/magic/clojure.spec.alpha.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.stacktrace.clj.dll b/magic-unity/Runtime/magic/clojure.stacktrace.clj.dll index 7a852290c..7d62680e3 100755 Binary files a/magic-unity/Runtime/magic/clojure.stacktrace.clj.dll and b/magic-unity/Runtime/magic/clojure.stacktrace.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.string.clj.dll b/magic-unity/Runtime/magic/clojure.string.clj.dll index f254ee423..fc729783a 100755 Binary files a/magic-unity/Runtime/magic/clojure.string.clj.dll and b/magic-unity/Runtime/magic/clojure.string.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.test.clj.dll b/magic-unity/Runtime/magic/clojure.test.clj.dll index 2ad49cb15..3affaec2f 100755 Binary files a/magic-unity/Runtime/magic/clojure.test.clj.dll and b/magic-unity/Runtime/magic/clojure.test.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.walk.clj.dll b/magic-unity/Runtime/magic/clojure.walk.clj.dll index ac8893e6f..85d625124 100755 Binary files a/magic-unity/Runtime/magic/clojure.walk.clj.dll and b/magic-unity/Runtime/magic/clojure.walk.clj.dll differ diff --git a/magic-unity/Runtime/magic/clojure.zip.clj.dll b/magic-unity/Runtime/magic/clojure.zip.clj.dll index f869e54ba..df43e198c 100755 Binary files a/magic-unity/Runtime/magic/clojure.zip.clj.dll and b/magic-unity/Runtime/magic/clojure.zip.clj.dll differ diff --git a/magic-unity/package.json b/magic-unity/package.json index 9d3c975f6..8ecc87901 100644 --- a/magic-unity/package.json +++ b/magic-unity/package.json @@ -1,6 +1,6 @@ { "name": "sg.flybot.magic.unity", - "version": "0.12.1", + "version": "0.13.0", "displayName": "MAGIC Unity Integration", "description": "The integration of the MAGIC Clojure compiler into Unity. MAGIC was created by Ramsey Nasser. This package is Flybot's fork.", "unity": "2021.2", diff --git a/nostrand/README.md b/nostrand/README.md index ef4e742f1..bb6e335cf 100644 --- a/nostrand/README.md +++ b/nostrand/README.md @@ -150,7 +150,7 @@ A project that pins its dependencies as git submodules can treat `.gitmodules` a ### Build and test tasks -`nos build` and `nos test` ship with the host, so most projects need no task file at all. Both derive the namespaces they work on by scanning the project's own source paths, and an optional `magic.edn` at the root states whatever differs from the defaults. +`nos build` and `nos test` ship with the host, so most projects need no task file at all. Both derive the namespaces they work on by scanning the project's own source paths, and an optional `magic.edn` at the root states whatever differs from the defaults. `nos build` also copies the C# assemblies a dependency ships into the output, so a library's hand-written C# travels with the Clojure that imports it ([a library's C# assembly](../docs/native-assemblies.md)). The pieces they are made of are public in `nostrand.tasks`, so a custom task composes them instead of restating them: `compile-project`, `run-clojure-tests`, `project-namespaces`, and the `production-flags` and `test-flags` maps. [The `nos` CLI](../docs/nos-cli.md) is the full reference for the tasks, the `magic.edn` keys, and the compiler flags each one binds. diff --git a/nostrand/nostrand/core.clj b/nostrand/nostrand/core.clj index 4c67a285b..53a12a6f0 100644 --- a/nostrand/nostrand/core.clj +++ b/nostrand/nostrand/core.clj @@ -5,7 +5,7 @@ (:require [clojure.string :as string] [nostrand.deps.basis :as basis] [nostrand.deps.submodules :as submodules]) - (:import [System.IO Path File])) + (:import [System.IO Directory Path File])) (def -assembly-path (atom (string/split (or (Environment/GetEnvironmentVariable "MONO_PATH") ".") @@ -15,6 +15,9 @@ (atom (string/split (or (Environment/GetEnvironmentVariable "CLOJURE_LOAD_PATH") ".") (re-pattern (str Path/PathSeparator))))) +(defn- absolute-load-path [] + (mapv #(Path/GetFullPath %) @-load-path)) + (defn resolve-assembly-load [asm] (let [candidates (for [prefix @-assembly-path ext ["" ".dll" ".exe"]] @@ -26,7 +29,7 @@ (assembly-load-from full-asm-path)))) (defn update-load-path [] - (let [abs-paths (mapv #(System.IO.Path/GetFullPath %) @-load-path)] + (let [abs-paths (absolute-load-path)] ;; CLOJURE_LOAD_PATH gets absolute roots (like *load-paths*), so a loader ;; scanning it finds files from any cwd, matching ClojureCLR. (Environment/SetEnvironmentVariable @@ -59,6 +62,37 @@ (doseq [p paths] (add-assembly-path p))) +(defn- dir-prefix + "dir lower-cased and separator-terminated. The separator stops a prefix test + matching a sibling; the case folding is for macOS and Windows." + [dir] + (let [sep (str Path/DirectorySeparatorChar)] + (string/lower-case (cond-> dir (not (string/ends-with? dir sep)) (str sep))))) + +(defn- under-any? [prefixes file] + (let [path (string/lower-case (Path/GetFullPath file))] + (boolean (some #(string/starts-with? path %) prefixes)))) + +(defn- assembly-file [asm] + (when-not (.IsDynamic asm) + (not-empty (.Location asm)))) + +(defn loaded-assembly-files + "The files of the assemblies this process loaded from under a source path. + The current directory sits on the load path to resolve task files, not as a + source path, so matching it would take in every assembly beneath it." + [] + (let [cwd (dir-prefix (Path/GetFullPath ".")) + prefixes (->> (absolute-load-path) + distinct + (filter #(Directory/Exists %)) + (map dir-prefix) + (remove #(= cwd %)))] + (for [asm (.GetAssemblies AppDomain/CurrentDomain) + :let [file (assembly-file asm)] + :when (and file (under-any? prefixes file))] + file))) + (defn reference* [asms] (doseq [asm asms] (let [a (str asm)] diff --git a/nostrand/nostrand/repl.clj b/nostrand/nostrand/repl.clj index bbaa0a5ba..4d64032a0 100644 --- a/nostrand/nostrand/repl.clj +++ b/nostrand/nostrand/repl.clj @@ -35,7 +35,7 @@ :as args}] (binding [*ns* (find-ns 'user) *warn-on-reflection* *warn-on-reflection* - *unchecked-math* *warn-on-reflection*] + *unchecked-math* *unchecked-math*] (loop [s (.Edit line-editor (prompt) "")] (when s (if-not (balanced? s) @@ -85,7 +85,7 @@ (binding [*ns* (find-ns 'user) *out* (StringWriter. sb) *warn-on-reflection* *warn-on-reflection* - *unchecked-math* *warn-on-reflection*] + *unchecked-math* *unchecked-math*] (loop [running @socket-repl-running] (when running (try diff --git a/nostrand/nostrand/tasks.clj b/nostrand/nostrand/tasks.clj index abe62450f..7f997c4b3 100644 --- a/nostrand/nostrand/tasks.clj +++ b/nostrand/nostrand/tasks.clj @@ -6,6 +6,7 @@ (:import [Nostrand Nostrand] [System.IO Directory File Path] + [System.Security.Cryptography MD5] [System.Threading Thread ThreadStart] [System.Reflection AssemblyInformationalVersionAttribute]) (:require [nostrand.repl :as repl] @@ -145,7 +146,7 @@ `compile-project` / `run-clojure-tests`. A plain map, so a task that needs to deviate assoc's onto it or passes its own (see `test-flags`). Shared so consumer dotnet.clj tasks do not each restate the set." - {#'*unchecked-math* true + {#'*unchecked-math* false #'*warn-on-reflection* true #'mflags/*strongly-typed-invokes* true #'mflags/*direct-linking* true @@ -202,21 +203,77 @@ (paths-namespaces (:paths (or basis (basis/create-basis (basis/project-deps-file) (vec aliases)))))) (remove (set exclude)))) +(def ^:private clj-assembly-suffixes [".clj.dll" ".cljc.dll" ".cljr.dll"]) + +(defn- clj-assembly? [file] + (let [file-name (string/lower-case (Path/GetFileName file))] + (boolean (some #(string/ends-with? file-name %) clj-assembly-suffixes)))) + +(defn- content-hash [file] + (with-open [stream (File/OpenRead file) + md5 (MD5/Create)] + (Convert/ToBase64String (.ComputeHash md5 stream)))) + +(defn- same-content? [a b] + (and (File/Exists b) + (= (content-hash a) (content-hash b)))) + +(defn- reject-name-clash! [files] + (doseq [[file-name group] (group-by #(Path/GetFileName %) files) + :when (and (next group) + (next (distinct (map content-hash group))))] + (throw (ex-info (str "Assemblies named " file-name " differ, so only one can" + " reach the build output:\n " + (string/join "\n " (sort group))) + {:file-name file-name :files (sort group)})))) + +(defn- report-stale! [out shipped] + (when (Directory/Exists out) + (doseq [file (sort (Directory/GetFiles out "*.dll")) + :let [file-name (Path/GetFileName file)] + :when (and (not (clj-assembly? file)) + (not (shipped file-name)))] + (println "No dependency ships" file-name "any more; delete it from" out)))) + +(defn- pending-copies [out files] + (for [file files + :let [dest (Path/Combine out (Path/GetFileName file))] + :when (and (not= (Path/GetFullPath file) (Path/GetFullPath dest)) + (not (same-content? file dest)))] + [file dest])) + +(defn- copy-csharp-assemblies! + "Copy into out the assemblies the build loaded that MAGIC did not compile." + [out] + (let [files (vec (remove clj-assembly? (nos/loaded-assembly-files)))] + (reject-name-clash! files) + (when-let [copies (seq (pending-copies out files))] + (Directory/CreateDirectory out) + (doseq [[file dest] copies] + (println "Copying C# assembly" (Path/GetFileName file)) + (File/Copy file dest true))) + (report-stale! out (set (map #(Path/GetFileName %) files))))) + (defn compile-project "Compile a project's namespaces (and their transitive requires) into a DLL - dir. With no :namespaces the set is derived from the source paths :aliases - contributes (see `project-namespaces`); pass :namespaces to state it instead - (e.g. a single root, or a vendored lib not reachable by `require`). Options: + dir, then copy in the C# assemblies the build loaded, so a library's C# + travels with the namespaces that import it. With no :namespaces the set is + derived from the source paths :aliases contributes (see `project-namespaces`); + pass :namespaces to state it instead (e.g. a single root, or a lib not + reachable by `require`). Options: :namespaces explicit namespaces to compile (overrides derivation) :exclude namespaces to drop from the set :aliases deps.edn aliases to activate, e.g. [:clr] :out *compile-path* (default \"build\") + :csharp-out where the C# assemblies land (default :out). :clean? only ever + empties :out, so a copy whose library left the deps stays until + you delete it; the build names it rather than deleting it :clean? wipe :out first (default false; for Unity dirs that must not carry stale DLLs) :flags var->value binding map (default `production-flags`) Drop-in for a consumer's `nos dotnet/build`: (defn build [] (tasks/compile-project :aliases [:clr]))" - [& {:keys [namespaces exclude aliases out clean? flags] + [& {:keys [namespaces exclude aliases out csharp-out clean? flags] :or {out "build" clean? false flags production-flags}}] (let [basis (when (seq aliases) (nos/establish-deps-edn (basis/project-deps-file) aliases)) nses (task-namespaces namespaces aliases exclude basis)] @@ -228,6 +285,7 @@ (doseq [ns nses] (println "Compiling" ns) (compile ns)) + (copy-csharp-assemblies! (or csharp-out out)) (println "Done.")))) (defn run-clojure-tests @@ -279,11 +337,12 @@ (s/def ::exclude (s/coll-of symbol? :kind vector?)) (s/def ::re string?) (s/def ::out string?) +(s/def ::csharp-out string?) (s/def ::clean? boolean?) (s/def ::flags (s/map-of qualified-symbol? any?)) (s/def ::exclude-vars (s/coll-of qualified-symbol? :kind vector?)) -(s/def ::build (s/keys :opt-un [::aliases ::namespaces ::exclude ::out ::clean? ::flags])) +(s/def ::build (s/keys :opt-un [::aliases ::namespaces ::exclude ::out ::csharp-out ::clean? ::flags])) (s/def ::test (s/keys :opt-un [::aliases ::namespaces ::exclude ::re ::flags ::exclude-vars])) (s/def ::magic-edn (s/keys :opt-un [::build ::test])) @@ -311,13 +370,14 @@ (defn build "Compile the project under MAGIC, per magic.edn :build. Run as `nos build`." [] - (let [{:keys [aliases namespaces exclude out clean? flags] + (let [{:keys [aliases namespaces exclude out csharp-out clean? flags] :or {out "build" clean? true}} (:build (read-magic-edn))] (compile-project :aliases (vec aliases) :namespaces namespaces :exclude exclude :out out + :csharp-out csharp-out :clean? clean? :flags (resolve-flags production-flags flags)))) diff --git a/nostrand/references/clojure.clr.io.clj.dll b/nostrand/references/clojure.clr.io.clj.dll index 38b618c91..70efbdd67 100755 Binary files a/nostrand/references/clojure.clr.io.clj.dll and b/nostrand/references/clojure.clr.io.clj.dll differ diff --git a/nostrand/references/clojure.clr.shell.clj.dll b/nostrand/references/clojure.clr.shell.clj.dll index 14f3ebb66..1916b3f26 100755 Binary files a/nostrand/references/clojure.clr.shell.clj.dll and b/nostrand/references/clojure.clr.shell.clj.dll differ diff --git a/nostrand/references/clojure.core.clj.dll b/nostrand/references/clojure.core.clj.dll index d5b6d2f3e..d273c1819 100755 Binary files a/nostrand/references/clojure.core.clj.dll and b/nostrand/references/clojure.core.clj.dll differ diff --git a/nostrand/references/clojure.core.protocols.clj.dll b/nostrand/references/clojure.core.protocols.clj.dll index ec123614d..77b52751c 100755 Binary files a/nostrand/references/clojure.core.protocols.clj.dll and b/nostrand/references/clojure.core.protocols.clj.dll differ diff --git a/nostrand/references/clojure.core.reducers.clj.dll b/nostrand/references/clojure.core.reducers.clj.dll index 52241e7a9..354226dc1 100755 Binary files a/nostrand/references/clojure.core.reducers.clj.dll and b/nostrand/references/clojure.core.reducers.clj.dll differ diff --git a/nostrand/references/clojure.core.server.clj.dll b/nostrand/references/clojure.core.server.clj.dll index 56840382f..b96151394 100755 Binary files a/nostrand/references/clojure.core.server.clj.dll and b/nostrand/references/clojure.core.server.clj.dll differ diff --git a/nostrand/references/clojure.core.specs.alpha.clj.dll b/nostrand/references/clojure.core.specs.alpha.clj.dll index c04adee32..d6922127b 100755 Binary files a/nostrand/references/clojure.core.specs.alpha.clj.dll and b/nostrand/references/clojure.core.specs.alpha.clj.dll differ diff --git a/nostrand/references/clojure.core_clr.clj.dll b/nostrand/references/clojure.core_clr.clj.dll index 96d02d4e4..92cb5822d 100755 Binary files a/nostrand/references/clojure.core_clr.clj.dll and b/nostrand/references/clojure.core_clr.clj.dll differ diff --git a/nostrand/references/clojure.core_deftype.clj.dll b/nostrand/references/clojure.core_deftype.clj.dll index a7776b071..62b6e82d6 100755 Binary files a/nostrand/references/clojure.core_deftype.clj.dll and b/nostrand/references/clojure.core_deftype.clj.dll differ diff --git a/nostrand/references/clojure.core_print.clj.dll b/nostrand/references/clojure.core_print.clj.dll index e03552598..2ad29b2b8 100755 Binary files a/nostrand/references/clojure.core_print.clj.dll and b/nostrand/references/clojure.core_print.clj.dll differ diff --git a/nostrand/references/clojure.core_proxy.clj.dll b/nostrand/references/clojure.core_proxy.clj.dll index ed046a63a..391f7e4bb 100755 Binary files a/nostrand/references/clojure.core_proxy.clj.dll and b/nostrand/references/clojure.core_proxy.clj.dll differ diff --git a/nostrand/references/clojure.data.clj.dll b/nostrand/references/clojure.data.clj.dll index f7a4f5a90..eacf5684c 100755 Binary files a/nostrand/references/clojure.data.clj.dll and b/nostrand/references/clojure.data.clj.dll differ diff --git a/nostrand/references/clojure.instant.clj.dll b/nostrand/references/clojure.instant.clj.dll index 0effa716a..87641a937 100755 Binary files a/nostrand/references/clojure.instant.clj.dll and b/nostrand/references/clojure.instant.clj.dll differ diff --git a/nostrand/references/clojure.main.clj.dll b/nostrand/references/clojure.main.clj.dll index fb60098d6..2b0a4e3fe 100755 Binary files a/nostrand/references/clojure.main.clj.dll and b/nostrand/references/clojure.main.clj.dll differ diff --git a/nostrand/references/clojure.pprint.cl_format.clj.dll b/nostrand/references/clojure.pprint.cl_format.clj.dll index 76f705cc0..b5919311a 100755 Binary files a/nostrand/references/clojure.pprint.cl_format.clj.dll and b/nostrand/references/clojure.pprint.cl_format.clj.dll differ diff --git a/nostrand/references/clojure.pprint.column_writer.clj.dll b/nostrand/references/clojure.pprint.column_writer.clj.dll index 40216b156..32ac7091c 100755 Binary files a/nostrand/references/clojure.pprint.column_writer.clj.dll and b/nostrand/references/clojure.pprint.column_writer.clj.dll differ diff --git a/nostrand/references/clojure.pprint.dispatch.clj.dll b/nostrand/references/clojure.pprint.dispatch.clj.dll index f7fea81c7..708aed855 100755 Binary files a/nostrand/references/clojure.pprint.dispatch.clj.dll and b/nostrand/references/clojure.pprint.dispatch.clj.dll differ diff --git a/nostrand/references/clojure.pprint.pprint_base.clj.dll b/nostrand/references/clojure.pprint.pprint_base.clj.dll index de706ed98..2c3c1cb25 100755 Binary files a/nostrand/references/clojure.pprint.pprint_base.clj.dll and b/nostrand/references/clojure.pprint.pprint_base.clj.dll differ diff --git a/nostrand/references/clojure.pprint.pretty_writer.clj.dll b/nostrand/references/clojure.pprint.pretty_writer.clj.dll index 662c180bc..6385f706d 100755 Binary files a/nostrand/references/clojure.pprint.pretty_writer.clj.dll and b/nostrand/references/clojure.pprint.pretty_writer.clj.dll differ diff --git a/nostrand/references/clojure.pprint.print_table.clj.dll b/nostrand/references/clojure.pprint.print_table.clj.dll index b59845147..01fd2a9df 100755 Binary files a/nostrand/references/clojure.pprint.print_table.clj.dll and b/nostrand/references/clojure.pprint.print_table.clj.dll differ diff --git a/nostrand/references/clojure.pprint.utilities.clj.dll b/nostrand/references/clojure.pprint.utilities.clj.dll index 29314a830..f9eee78ac 100755 Binary files a/nostrand/references/clojure.pprint.utilities.clj.dll and b/nostrand/references/clojure.pprint.utilities.clj.dll differ diff --git a/nostrand/references/clojure.repl.clj.dll b/nostrand/references/clojure.repl.clj.dll index 6970b61c1..b935fee12 100755 Binary files a/nostrand/references/clojure.repl.clj.dll and b/nostrand/references/clojure.repl.clj.dll differ diff --git a/nostrand/references/clojure.set.clj.dll b/nostrand/references/clojure.set.clj.dll index b5553f54f..3166d10d6 100755 Binary files a/nostrand/references/clojure.set.clj.dll and b/nostrand/references/clojure.set.clj.dll differ diff --git a/nostrand/references/clojure.spec.alpha.clj.dll b/nostrand/references/clojure.spec.alpha.clj.dll index 3611c17d8..1e74276a7 100755 Binary files a/nostrand/references/clojure.spec.alpha.clj.dll and b/nostrand/references/clojure.spec.alpha.clj.dll differ diff --git a/nostrand/references/clojure.stacktrace.clj.dll b/nostrand/references/clojure.stacktrace.clj.dll index 7a852290c..7d62680e3 100755 Binary files a/nostrand/references/clojure.stacktrace.clj.dll and b/nostrand/references/clojure.stacktrace.clj.dll differ diff --git a/nostrand/references/clojure.string.clj.dll b/nostrand/references/clojure.string.clj.dll index f254ee423..fc729783a 100755 Binary files a/nostrand/references/clojure.string.clj.dll and b/nostrand/references/clojure.string.clj.dll differ diff --git a/nostrand/references/clojure.test.clj.dll b/nostrand/references/clojure.test.clj.dll index 2ad49cb15..3affaec2f 100755 Binary files a/nostrand/references/clojure.test.clj.dll and b/nostrand/references/clojure.test.clj.dll differ diff --git a/nostrand/references/clojure.tools.analyzer.ast.clj.dll b/nostrand/references/clojure.tools.analyzer.ast.clj.dll index 9cd4563b0..bbf7cf375 100755 Binary files a/nostrand/references/clojure.tools.analyzer.ast.clj.dll and b/nostrand/references/clojure.tools.analyzer.ast.clj.dll differ diff --git a/nostrand/references/clojure.tools.analyzer.clj.dll b/nostrand/references/clojure.tools.analyzer.clj.dll index cba1a74d0..92269d07b 100755 Binary files a/nostrand/references/clojure.tools.analyzer.clj.dll and b/nostrand/references/clojure.tools.analyzer.clj.dll differ diff --git a/nostrand/references/clojure.tools.analyzer.passes.clj.dll b/nostrand/references/clojure.tools.analyzer.passes.clj.dll index 063ca7dd2..498063afd 100755 Binary files a/nostrand/references/clojure.tools.analyzer.passes.clj.dll and b/nostrand/references/clojure.tools.analyzer.passes.clj.dll differ diff --git a/nostrand/references/clojure.tools.analyzer.utils.clj.dll b/nostrand/references/clojure.tools.analyzer.utils.clj.dll index 2d958b9aa..e6f10388d 100755 Binary files a/nostrand/references/clojure.tools.analyzer.utils.clj.dll and b/nostrand/references/clojure.tools.analyzer.utils.clj.dll differ diff --git a/nostrand/references/clojure.walk.clj.dll b/nostrand/references/clojure.walk.clj.dll index ac8893e6f..85d625124 100755 Binary files a/nostrand/references/clojure.walk.clj.dll and b/nostrand/references/clojure.walk.clj.dll differ diff --git a/nostrand/references/clojure.zip.clj.dll b/nostrand/references/clojure.zip.clj.dll index f869e54ba..df43e198c 100755 Binary files a/nostrand/references/clojure.zip.clj.dll and b/nostrand/references/clojure.zip.clj.dll differ diff --git a/nostrand/references/mage.core.clj.dll b/nostrand/references/mage.core.clj.dll index 1ab4f160e..c81db7c79 100755 Binary files a/nostrand/references/mage.core.clj.dll and b/nostrand/references/mage.core.clj.dll differ diff --git a/nostrand/references/magic.analyzer.clj.dll b/nostrand/references/magic.analyzer.clj.dll index a467259cd..a657c96c2 100755 Binary files a/nostrand/references/magic.analyzer.clj.dll and b/nostrand/references/magic.analyzer.clj.dll differ diff --git a/nostrand/references/magic.analyzer.collect_closed_overs.clj.dll b/nostrand/references/magic.analyzer.collect_closed_overs.clj.dll index 3151f7d30..fd9a4fcb7 100755 Binary files a/nostrand/references/magic.analyzer.collect_closed_overs.clj.dll and b/nostrand/references/magic.analyzer.collect_closed_overs.clj.dll differ diff --git a/nostrand/references/magic.analyzer.literal_reinterpretation.clj.dll b/nostrand/references/magic.analyzer.literal_reinterpretation.clj.dll index 5ace5203c..42d91c66a 100755 Binary files a/nostrand/references/magic.analyzer.literal_reinterpretation.clj.dll and b/nostrand/references/magic.analyzer.literal_reinterpretation.clj.dll differ diff --git a/nostrand/references/magic.analyzer.typed_passes.clj.dll b/nostrand/references/magic.analyzer.typed_passes.clj.dll index 5330b3f06..494cc8b10 100755 Binary files a/nostrand/references/magic.analyzer.typed_passes.clj.dll and b/nostrand/references/magic.analyzer.typed_passes.clj.dll differ diff --git a/nostrand/references/magic.analyzer.uniquify.clj.dll b/nostrand/references/magic.analyzer.uniquify.clj.dll index 9b75ea967..53ba7ad2c 100755 Binary files a/nostrand/references/magic.analyzer.uniquify.clj.dll and b/nostrand/references/magic.analyzer.uniquify.clj.dll differ diff --git a/nostrand/references/magic.analyzer.untyped_passes.clj.dll b/nostrand/references/magic.analyzer.untyped_passes.clj.dll index 5e2197a76..722f23824 100755 Binary files a/nostrand/references/magic.analyzer.untyped_passes.clj.dll and b/nostrand/references/magic.analyzer.untyped_passes.clj.dll differ diff --git a/nostrand/references/magic.api.clj.dll b/nostrand/references/magic.api.clj.dll index 67935bd3d..af9091bb6 100755 Binary files a/nostrand/references/magic.api.clj.dll and b/nostrand/references/magic.api.clj.dll differ diff --git a/nostrand/references/magic.core.clj.dll b/nostrand/references/magic.core.clj.dll index 7c6cb46f4..376a4d642 100755 Binary files a/nostrand/references/magic.core.clj.dll and b/nostrand/references/magic.core.clj.dll differ diff --git a/nostrand/references/magic.interop.clj.dll b/nostrand/references/magic.interop.clj.dll index 338ac7ce7..85d441e11 100755 Binary files a/nostrand/references/magic.interop.clj.dll and b/nostrand/references/magic.interop.clj.dll differ diff --git a/nostrand/references/magic.intrinsics.clj.dll b/nostrand/references/magic.intrinsics.clj.dll index aa7bd76eb..a65e8c67a 100755 Binary files a/nostrand/references/magic.intrinsics.clj.dll and b/nostrand/references/magic.intrinsics.clj.dll differ diff --git a/unity-examples/magic-unity-coexist/Assets/Editor/CoexistenceProbe.cs b/unity-examples/magic-unity-coexist/Assets/Editor/CoexistenceProbe.cs index 921608618..7c74294ee 100644 --- a/unity-examples/magic-unity-coexist/Assets/Editor/CoexistenceProbe.cs +++ b/unity-examples/magic-unity-coexist/Assets/Editor/CoexistenceProbe.cs @@ -23,6 +23,9 @@ public static class CoexistenceProbe // The same extensions as in Magic.Unity's PlayerCljAssemblies static readonly string[] Extensions = { ".clj", ".cljc", ".cljr" }; + // The unconstrained plugin, expected in both Editor states. + const string CsharpName = "smoke_csharp"; + static bool IsCljAssembly(string name, string suffix) { return Extensions.Any(e => name.EndsWith(e + suffix, StringComparison.OrdinalIgnoreCase)); @@ -30,6 +33,10 @@ static bool IsCljAssembly(string name, string suffix) public static void Run() { + var csharpInDomain = AppDomain + .CurrentDomain.GetAssemblies() + .Any(a => a.GetName().Name == CsharpName); + var preloaded = AppDomain .CurrentDomain.GetAssemblies() .Select(a => a.GetName().Name) @@ -67,17 +74,32 @@ public static void Run() + $"core-clj-load={loadDetail} " + $"clojure-versions=[{string.Join(",", clojureVersions)}] " + $"editor-clj-refs={CljReferences(AssembliesType.Editor)} " - + $"player-clj-refs={CljReferences(AssembliesType.PlayerWithoutTestAssemblies)}" + + $"player-clj-refs={CljReferences(AssembliesType.PlayerWithoutTestAssemblies)} " + + $"csharp-in-domain={csharpInDomain.ToString().ToLowerInvariant()} " + + $"csharp-editor-refs={CsharpReferences(AssembliesType.Editor)}" ); } - static int CljReferences(AssembliesType type) + static int ReferenceCount(AssembliesType type, Func matches) { return CompilationPipeline .GetAssemblies(type) .SelectMany(a => a.allReferences) - .Where(r => IsCljAssembly(r, ".dll")) + .Where(matches) .Distinct(StringComparer.OrdinalIgnoreCase) .Count(); } + + static int CsharpReferences(AssembliesType type) + { + return ReferenceCount( + type, + r => r.EndsWith("/" + CsharpName + ".dll", StringComparison.OrdinalIgnoreCase) + ); + } + + static int CljReferences(AssembliesType type) + { + return ReferenceCount(type, r => IsCljAssembly(r, ".dll")); + } } diff --git a/unity-examples/magic-unity-coexist/Assets/Plugins/Consumer/smoke_csharp.dll b/unity-examples/magic-unity-coexist/Assets/Plugins/Consumer/smoke_csharp.dll new file mode 100644 index 000000000..559ecb267 Binary files /dev/null and b/unity-examples/magic-unity-coexist/Assets/Plugins/Consumer/smoke_csharp.dll differ diff --git a/unity-examples/magic-unity-coexist/README.md b/unity-examples/magic-unity-coexist/README.md index fbaf6e702..5d7c15610 100644 --- a/unity-examples/magic-unity-coexist/README.md +++ b/unity-examples/magic-unity-coexist/README.md @@ -49,20 +49,24 @@ Two ingredients smoke lacks: 1. **An immutable (PackageCache) install.** `bb coexist-noise` installs from a repacked tarball; `magic-unity-smoke` is a mutable `file:` install, on which this bug class cannot appear. -2. **Consumer-compiled Clojure DLLs outside the package.** - `Assets/Plugins/Consumer/` stands in for a consumer's own compiled - namespaces, with one DLL of each load shape: `smoke.control_flow.clj.dll` - has fn types implementing `Magic.Function` (typed invoke), so unconstrained - in a ClojureCLR Editor it fails type load and Unity unloads it as broken -- - the loud symptom; `smoke.interop.cljr.dll` has none, loads cleanly against - ClojureCLR's `Clojure.dll`, and covers the silent-bind shape. Their `.meta`s - are package *output*, written by the constrainer on import, so the metas are - gitignored and deleted before each import; committing them would turn the - constraining into setup. +2. **DLLs a consumer compiled, sitting outside the package**, in + `Assets/Plugins/Consumer/`, one per load shape. The ClojureCLR runtime comes from the package, exactly as a consumer's would, which is why the project runs API Compatibility Level `.NET Framework`. +### The three load shapes + +| `Assets/Plugins/Consumer/` | What it holds | What it pins | +|---|---|---| +| `smoke.control_flow.clj.dll` | fn types implementing `Magic.Function`, so typed invoke | unconstrained in a ClojureCLR Editor it fails type load and Unity unloads it as broken, the loud symptom | +| `smoke.interop.cljr.dll` | no `Magic.Function` types | it binds cleanly to ClojureCLR's `Clojure.dll`, the silent shape | +| `smoke_csharp.dll` | plain C#, the kind a library ships | the constrainer matches the three compiled-Clojure extensions only, so it walks past this one and both Editor states load it | + +The `.meta`s are package output, written by the constrainer on import, so they +are gitignored and deleted before each import. Committing them would turn the +constraining into setup. + ## Running it Close all open instances of Unity first, then run one of: @@ -83,7 +87,7 @@ The `[CoexistenceProbe]` marker line in the editor log carries the per-run state: ``` -[CoexistenceProbe] symbol=unset preloaded-clj=0 core-clj-loadable=false core-clj-load=FileNotFoundException clojure-versions=[1.11.0.0] editor-clj-refs=0 player-clj-refs=39 +[CoexistenceProbe] symbol=unset preloaded-clj=0 core-clj-loadable=false core-clj-load=FileNotFoundException clojure-versions=[1.11.0.0] editor-clj-refs=0 player-clj-refs=39 csharp-in-domain=true csharp-editor-refs=1 ``` Player builds are unaffected by the selection; the end-to-end confirmation is diff --git a/unity-examples/magic-unity-smoke/.gitignore b/unity-examples/magic-unity-smoke/.gitignore index 056f791ce..520fe5fe6 100644 --- a/unity-examples/magic-unity-smoke/.gitignore +++ b/unity-examples/magic-unity-smoke/.gitignore @@ -24,7 +24,7 @@ UserSettings/ *.opendb *.VC.db -# MAGIC compiler output. `nos dotnet/build` writes here. The MAGIC +# MAGIC compiler output. `nos build` writes here. The MAGIC # compiler window defaults to Assets/Compiled instead, which we # also ignore. Assets/Plugins/Magic/ diff --git a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/csharp.clj b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/csharp.clj new file mode 100644 index 000000000..7c3cc790c --- /dev/null +++ b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/csharp.clj @@ -0,0 +1,17 @@ +(ns smoke.csharp + "Calls into a C# assembly a library ships, which IL2CPP has to keep callable + like any other interop." + (:require [smoke.check :refer [check]] + [smoke-csharp.load-dll]) + (:import [smoke_csharp Greeter])) + +(defn suite [] + [(check "a shipped assembly's static method is callable" + #(Greeter/Greet "magic") + "hello, magic") + (check "a shipped assembly's value-typed args stay unboxed" + #(Greeter/Add 2 3) + 5) + (check "a shipped assembly's const field reads" + (fn [] Greeter/Marker) + "smoke-csharp-v1")]) diff --git a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/csharp.clj.meta b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/csharp.clj.meta new file mode 100644 index 000000000..05bb71e05 --- /dev/null +++ b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/csharp.clj.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: ed36d7ded606740a4af651c1ba32a19c +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/numeric_casts.clj b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/numeric_casts.clj new file mode 100644 index 000000000..83fda9db4 --- /dev/null +++ b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/numeric_casts.clj @@ -0,0 +1,36 @@ +(ns smoke.numeric-casts + "Narrowing conversions under AOT, one check per compiled shape. + + A cast that cannot hold its value calls the RT cast and throws, an + in-range one returns, a value-preserving one keeps its plain conv + opcode, and unchecked-int keeps calling the RT unchecked cast. The + array index, the aset value and an interop parameter each narrow at + their own call site." + (:require [smoke.check :refer [check]])) + +(defn- threw? [thunk] + (try (thunk) :no-throw (catch ArgumentException e :threw))) + +(defn suite [] + [(check "out-of-range cast throws" + #(threw? (fn [] ((fn [^long x] (int x)) 4294967296))) :threw) + (check "in-range cast converts" + #((fn [^long x] (int x)) 7) 7) + (check "value-preserving conversion keeps plain conv" + #((fn [^int x] (long x)) 7) 7) + (check "unchecked cast calls the RT unchecked cast" + #((fn [^long x] (unchecked-int x)) 4294967296) 0) + (check "aget index narrowing throws" + #(threw? (fn [] (let [a (int-array [10 20 30])] + ((fn [^long i] (aget a i)) 4294967296)))) :threw) + (check "aset value narrowing throws" + #(threw? (fn [] (let [a (int-array [10 20 30])] + ((fn [^long v] (aset a 0 v)) 4294967296)))) :threw) + (check "interop parameter narrowing throws" + #(threw? (fn [] ((fn [^long i] (.Substring "hello" i)) 4294967296))) :threw) + (check "boxed ulong casts through longCast" + #(int (identity (ulong 1))) 1) + (check "fractional literal cast truncates" + #(int 1.5) 1) + (check "out-of-range literal cast throws at runtime" + #(threw? (fn [] (int 4294967296))) :threw)]) diff --git a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/numeric_casts.clj.meta b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/numeric_casts.clj.meta new file mode 100644 index 000000000..d782366f5 --- /dev/null +++ b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/numeric_casts.clj.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: cb6ec51c84344da1bb796f907114fdad +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/polymorphism.clj b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/polymorphism.clj index 2f44e3750..3b9cd4169 100644 --- a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/polymorphism.clj +++ b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/polymorphism.clj @@ -8,7 +8,9 @@ rejects at C++ compile time. The protocol-hinted parameter check is bound to the hint-relax fix: a ^Protocol param used to cast to the protocol's generated interface, which extend-protocol - implementers are not instances of. The rest of the suite is broad + implementers are not instances of. The IPersistentMap deftype is the + only shape here that emits two methods differing solely in return + type, which no C# source can express. The rest of the suite is broad construct coverage to catch dispatch-related regressions." (:require [smoke.check :refer [check]])) @@ -37,6 +39,16 @@ (add-item [this x] (set! items (conj items x)) this) (item-vec [_] items)) +(deftype MapBox [m] + clojure.lang.IPersistentMap + (count [_] (count m)) + (^clojure.lang.IPersistentCollection cons [_ e] (MapBox. (conj m e))) + (^clojure.lang.IPersistentMap assoc [_ k v] (MapBox. (assoc m k v))) + (^clojure.lang.Associative assoc [_ k v] (MapBox. (assoc m k v))) + (valAt [_ k] (get m k)) + (valAt [_ k nf] (get m k nf)) + (seq [_] (seq m))) + (defprotocol IStrLen (str-len [s])) @@ -107,6 +119,12 @@ (check "protocol-hinted param with extend-protocol implementer" #(hinted-len "hello") 5) + (check "deftype over IPersistentMap counts through the Counted slot" + #(count (MapBox. {:a 1 :b 2})) 2) + (check "return-type-hinted cons overload" + #(count (conj (MapBox. {:a 1}) [:b 2])) 2) + (check "return-type-hinted assoc overload" + #(get (assoc (MapBox. {:a 1}) :b 2) :b) 2) (check "multimethod :dog" #(animal-sound {:kind :dog}) "woof") (check "multimethod default" diff --git a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/runner.clj b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/runner.clj index 4a54874da..7a8dfd173 100644 --- a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/runner.clj +++ b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/runner.clj @@ -13,6 +13,8 @@ [smoke.interop :as interop] [smoke.read-print :as read-print] [smoke.compare :as compare-suite] + [smoke.csharp :as csharp] + [smoke.numeric-casts :as numeric-casts] [clojure.string :as str])) (defn- run [] @@ -24,7 +26,9 @@ ["stdlib-1.10" (stdlib-1-10/suite)] ["interop" (interop/suite)] ["read-print" (read-print/suite)] - ["compare" (compare-suite/suite)]] + ["compare" (compare-suite/suite)] + ["csharp" (csharp/suite)] + ["numeric-casts" (numeric-casts/suite)]] flat (for [[group results] groups r results] (assoc r :group group))] diff --git a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/stdlib_1_10.clj b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/stdlib_1_10.clj index a96c323d7..a31ef6da1 100644 --- a/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/stdlib_1_10.clj +++ b/unity-examples/magic-unity-smoke/Assets/Clojure/smoke/stdlib_1_10.clj @@ -6,7 +6,8 @@ which is the part most likely to break under IL2CPP AOT. Throwable->map walks the InnerException chain and reads stack frames via System.Diagnostics.StackTrace. ex-triage/ex-str exercise the - String.Join array overload and the Printf formatter under AOT." + String.Join array overload and the Printf formatter under AOT. sort reaches + the meta and with-meta Vars from inside clojure.core." (:require [smoke.check :refer [check]] [clojure.main :as cmain])) @@ -87,4 +88,7 @@ :from-meta) (check "extend-via-metadata falls through to extend table" #(smoke-via-meta {}) - :extend-table)]) + :extend-table) + (check "sort carries the collection's metadata" + #(meta (sort (with-meta [3 1 2] {:x 1}))) + {:x 1})]) diff --git a/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp.meta b/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp.meta new file mode 100644 index 000000000..98e569009 --- /dev/null +++ b/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp.meta @@ -0,0 +1,8 @@ +fileFormatVersion: 2 +guid: 5db0f3aee457a49b0a334142b41879c4 +folderAsset: yes +DefaultImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/smoke_csharp.dll b/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/smoke_csharp.dll new file mode 100644 index 000000000..559ecb267 Binary files /dev/null and b/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/smoke_csharp.dll differ diff --git a/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/smoke_csharp.dll.meta b/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/smoke_csharp.dll.meta new file mode 100644 index 000000000..02f7cd70e --- /dev/null +++ b/unity-examples/magic-unity-smoke/Assets/Plugins/CSharp/smoke_csharp.dll.meta @@ -0,0 +1,33 @@ +fileFormatVersion: 2 +guid: 7cd338bdc9edb4811ad0caab4174f5a1 +PluginImporter: + externalObjects: {} + serializedVersion: 2 + iconMap: {} + executionOrder: {} + defineConstraints: [] + isPreloaded: 0 + isOverridable: 0 + isExplicitlyReferenced: 0 + validateReferences: 1 + platformData: + - first: + Any: + second: + enabled: 1 + settings: {} + - first: + Editor: Editor + second: + enabled: 0 + settings: + DefaultValueInitialized: true + - first: + Windows Store Apps: WindowsStoreApps + second: + enabled: 0 + settings: + CPU: AnyCPU + userData: + assetBundleName: + assetBundleVariant: diff --git a/unity-examples/magic-unity-smoke/README.md b/unity-examples/magic-unity-smoke/README.md index 2878d0b4a..c52978bef 100644 --- a/unity-examples/magic-unity-smoke/README.md +++ b/unity-examples/magic-unity-smoke/README.md @@ -11,10 +11,10 @@ Runtime regression suite for MAGIC's IL2CPP output. Catches bugs that only repro ```bash cd unity-examples/magic-unity-smoke -nos dotnet/build +nos build ``` -That reads `deps.edn` + `dotnet.clj`, wipes `Assets/Plugins/Magic/`, and recompiles `smoke.runner` plus its transitive deps into that directory using the production compiler flags (`*direct-linking*`, `*strongly-typed-invokes*`). +That reads `deps.edn` + `magic.edn`, wipes `Assets/Plugins/Magic/`, and recompiles `smoke.runner` plus its transitive deps into that directory using the production compiler flags (`*direct-linking*`, `*strongly-typed-invokes*`). The C# assembly that `csharp-lib` ships lands in `Assets/Plugins/CSharp/`; the wipe covers `Assets/Plugins/Magic/` only. Then in Unity: @@ -22,7 +22,7 @@ Then in Unity: 2. Open `Assets/Smoke.unity` (the scene has one GameObject with `SmokeTestRunner.cs` attached). 3. Use **MAGIC → Smoke → Build & Run IL2CPP** to build and launch the player. The built player shows a green PASS / red FAIL panel and writes the same report to `Player.log`. -To re-run after a Clojure edit: rerun `nos dotnet/build`, then the menu item again. +To re-run after a Clojure edit: rerun `nos build`, then the menu item again. To run the same suites under Mono (no Unity round-trip): `nos dotnet/run-tests` from this directory. Catches regressions that surface independent of IL2CPP and exits non-zero on any failure. @@ -39,14 +39,42 @@ One namespace per edge-case family. Each exports `(suite)` returning a vector of | `smoke.read-print` | 12 | Floating point through the reader and the printer: out-of-range literals reading as infinity, and doubles and floats surviving `pr-str` then `read-string`. IL2CPP supplies its own `Double.ToString` and exception handling, so Mono does not cover this. | | `smoke.stdlib-1-10` | 11 | Clojure 1.10 stdlib surface: `symbol`, `read+string`, `PrintWriter-on`, `tap>`, `Throwable->map`, ex-triage, extend-via-metadata. | | `smoke.interop` | 2 | `by-ref` on a type-hinted local. Written as `.cljr`, so the source-extension handling is exercised too. | +| `smoke.intrinsics` | 5 | Intrinsic lowering, and the fallback when an intrinsic declines. | +| `smoke.compare` | 10 | `compare` and `sort` ordering by UTF-16 code unit rather than OS collation. | +| `smoke.csharp` | 3 | Calls into a C# assembly a library ships (see below). | -75 checks total. All green under Mono and Standalone Mac IL2CPP. +93 checks total. All green under Mono and Standalone Mac IL2CPP. + +## The C# assembly example + +`csharp-lib/` is a library in the shape [docs/native-assemblies.md](../../docs/native-assemblies.md) describes, consumed here as a `:local/root` dep: + +``` +csharp-lib/ + deps-clr.edn {:paths ["src" "src_classes"]} + Greeter.cs the source + src/smoke_csharp/load_dll.cljr the loader, scanning CLOJURE_LOAD_PATH + src_classes/smoke_csharp.dll the assembly, committed +``` + +Rebuild the assembly with the command it was built with, from `csharp-lib/`: + +```bash +csc -nologo -deterministic -optimize+ -target:library \ + -out:src_classes/smoke_csharp.dll Greeter.cs +``` + +Use that command as written. The flags and the compiler version are both part of the output bytes ([why](../../docs/deterministic-compilation.md#committing-an-assembly-you-compiled-yourself)); these come from Roslyn 3.9. The repo commits the result in three places, here plus `Assets/Plugins/CSharp/` and `magic-unity-coexist/Assets/Plugins/Consumer/`, so rebuild it and update all three; `bb check-drift` fails if they diverge. + +`smoke.csharp` requires the loader and imports `[smoke_csharp Greeter]`, so `nos build` resolves the types through the loader at compile time, then copies `smoke_csharp.dll` into `:csharp-out`. Unity imports it as a plain managed plugin: no define constraint applies, so both Editor runtimes load it and a player build carries it. + +`Assets/Plugins/CSharp/` is committed here, `.meta` files included, so a clone has the assembly before anyone runs `nos build`. That part is a choice ([the two plugin folders](../../docs/unity-integration.md#the-two-plugin-folders)); the folder split is not, because `:clean?` deletes `Assets/Plugins/Magic/` on every build and a plugin living there would be reimported under a new GUID each time. ## Adding a new edge case 1. Add the minimal repro to the matching `smoke/*.clj` (or a new namespace, then `:require` it from `smoke.runner`). 2. Express it as `(check "name" #(...) expected-value)`. The harness wraps each thunk in try/catch and pretty-prints failures. -3. `nos dotnet/build`, then **MAGIC → Smoke → Build & Run IL2CPP**. Confirm green. +3. `nos build`, then **MAGIC → Smoke → Build & Run IL2CPP**. Confirm green. 4. Commit the smoke case alongside the fix. Rules: diff --git a/unity-examples/magic-unity-smoke/csharp-lib/Greeter.cs b/unity-examples/magic-unity-smoke/csharp-lib/Greeter.cs new file mode 100644 index 000000000..2d4693bfc --- /dev/null +++ b/unity-examples/magic-unity-smoke/csharp-lib/Greeter.cs @@ -0,0 +1,22 @@ +using System.Reflection; + +[assembly: AssemblyVersion("1.0.0.0")] +[assembly: AssemblyFileVersion("1.0.0.0")] + +namespace smoke_csharp +{ + public static class Greeter + { + public const string Marker = "smoke-csharp-v1"; + + public static string Greet(string who) + { + return "hello, " + who; + } + + public static int Add(int a, int b) + { + return a + b; + } + } +} diff --git a/unity-examples/magic-unity-smoke/csharp-lib/deps-clr.edn b/unity-examples/magic-unity-smoke/csharp-lib/deps-clr.edn new file mode 100644 index 000000000..2bad58f86 --- /dev/null +++ b/unity-examples/magic-unity-smoke/csharp-lib/deps-clr.edn @@ -0,0 +1 @@ +{:paths ["src" "src_classes"]} diff --git a/unity-examples/magic-unity-smoke/csharp-lib/src/smoke_csharp/load_dll.cljr b/unity-examples/magic-unity-smoke/csharp-lib/src/smoke_csharp/load_dll.cljr new file mode 100644 index 000000000..73a631333 --- /dev/null +++ b/unity-examples/magic-unity-smoke/csharp-lib/src/smoke_csharp/load_dll.cljr @@ -0,0 +1,14 @@ +(ns smoke-csharp.load-dll + "Puts smoke_csharp.dll in the process before anything imports its types. + Finds nothing under Unity, where CLOJURE_LOAD_PATH is unset and the assembly + is loaded already." + (:require [clojure.string :as str]) + (:import [System.IO Path File])) + +(let [roots (some-> (Environment/GetEnvironmentVariable "CLOJURE_LOAD_PATH") + (str/split (re-pattern (str Path/PathSeparator))))] + (when-let [dll (some (fn [root] + (let [p (Path/Combine root "smoke_csharp.dll")] + (when (File/Exists p) p))) + roots)] + (assembly-load-from dll))) diff --git a/unity-examples/magic-unity-smoke/csharp-lib/src_classes/smoke_csharp.dll b/unity-examples/magic-unity-smoke/csharp-lib/src_classes/smoke_csharp.dll new file mode 100644 index 000000000..559ecb267 Binary files /dev/null and b/unity-examples/magic-unity-smoke/csharp-lib/src_classes/smoke_csharp.dll differ diff --git a/unity-examples/magic-unity-smoke/deps.edn b/unity-examples/magic-unity-smoke/deps.edn index f115a192d..308d54e12 100644 --- a/unity-examples/magic-unity-smoke/deps.edn +++ b/unity-examples/magic-unity-smoke/deps.edn @@ -1,2 +1,2 @@ {:paths ["Assets/Clojure"] - :deps {}} + :deps {sg.flybot/smoke-csharp {:local/root "csharp-lib"}}} diff --git a/unity-examples/magic-unity-smoke/dotnet.clj b/unity-examples/magic-unity-smoke/dotnet.clj index cdfd2c2c6..2cd74e679 100644 --- a/unity-examples/magic-unity-smoke/dotnet.clj +++ b/unity-examples/magic-unity-smoke/dotnet.clj @@ -1,32 +1,11 @@ (ns dotnet - "Compile and test the smoke project under MAGIC. + "Run the smoke suites under MAGIC. - Invoked from the repo root via `nos dotnet/build` and `nos dotnet/run-tests`. - - Shape: one root namespace, production compiler flags pinned - (`*direct-linking*`, `*strongly-typed-invokes*`, `*elide-meta*`), - transitive deps pulled in by `compile`. Output drops into - Assets/Plugins/Magic which the Unity project picks up automatically. - - `run-tests` exercises the smoke suites under Mono before opening Unity, so - pure-CLR (non-IL2CPP) regressions surface without a Unity round-trip." + Compiling is `nos build`, configured in magic.edn. This holds the one task + that has no built-in: the suites are not clojure.test, they are maps a + SmokeTestRunner MonoBehaviour reads, so `nos test` cannot drive them." (:require [nostrand.tasks :as tasks])) -(def root-namespaces - "Root namespaces to compile. Everything they `require` is compiled - transitively. Add a namespace here if the smoke runner needs to - load it directly." - '[smoke.runner]) - -(defn build - "nos dotnet/build - - Wipes Assets/Plugins/Magic and recompiles every root namespace - (and its transitive deps) into that folder using the same compiler - flags as production. Unity sees the new DLLs on next focus." - [] - (tasks/compile-project :namespaces root-namespaces :out "Assets/Plugins/Magic" :clean? true)) - (defn run-tests "nos dotnet/run-tests diff --git a/unity-examples/magic-unity-smoke/magic.edn b/unity-examples/magic-unity-smoke/magic.edn new file mode 100644 index 000000000..b888ec1a7 --- /dev/null +++ b/unity-examples/magic-unity-smoke/magic.edn @@ -0,0 +1,3 @@ +{:build {:namespaces [smoke.runner] + :out "Assets/Plugins/Magic" + :csharp-out "Assets/Plugins/CSharp"}} diff --git a/version.edn b/version.edn index 790cf0828..b4ca704a9 100644 --- a/version.edn +++ b/version.edn @@ -1 +1 @@ -{:version "0.12.1"} +{:version "0.13.0"}