Summary
unityctl test run executes and reports results even when the project currently has unresolved C# compilation errors. In that state Unity runs the test suite against the last successfully compiled assemblies (stale DLLs in Library/ScriptAssemblies), so the command can return a green/passing result — or a result with a different test count — that does not reflect the code on disk. There is no warning that the code under test never compiled.
This is a correctness/safety problem: a passing test run after an edit that doesn't compile is silently meaningless, and an agent or human reading the result will believe their change is verified when it was never built.
Expected behavior
test run should refuse to run (or at minimum loudly warn) when compilation has failed, consistent with how play enter and record start already behave:
Cannot enter play mode - compilation errors exist. Fix the errors and try again.
Proposed: return a COMPILATION_ERROR result (non-zero CLI exit) such as
Cannot run tests - compilation errors exist. Fix the errors and try again.
Actual behavior
Tests run against stale assemblies and the command reports success (or a stale test list), with no indication that the current sources failed to compile.
Repro
- Have a project with at least one edit-mode test that passes.
unityctl asset refresh → confirm a clean compile.
- Introduce a compile error in any C# script in an assembly that the tests depend on (e.g. add
this wont compile; to a runtime script).
unityctl asset refresh → reports the compilation error (correct).
unityctl test run → observe it still reports a passing/green result (running against the previously compiled assemblies) instead of failing on the pre-existing compile error.
Root cause
HandleTestRun (UnityCtl.UnityPackage/Editor/UnityCtlClient.cs, ~L1054) builds a TestRunnerApi filter and calls Execute(...) immediately. It never checks EditorUtility.scriptCompilationFailed (or EditorApplication.isCompiling) before running.
The Bridge has no precondition gate for test.run either — its CommandConfig (UnityCtl.Bridge/BridgeEndpoints.cs, ~L66) only sets a timeout and CompletionEvent = TestFinished. By contrast, the play-enter path checks compilation state and short-circuits with COMPILATION_ERROR (BridgeEndpoints.cs ~L523-534), and record start does the same (~L756). EditorUtility.scriptCompilationFailed is already used elsewhere in the plugin (UnityCtlClient.cs ~L1503), so the signal is readily available.
Suggested fix
Guard test run on EditorUtility.scriptCompilationFailed before executing — either in HandleTestRun (return an error result) or as a Bridge precondition mirroring the play-enter/record-start gate — so a stale assembly can never produce a misleading green result. Reuse the existing COMPILATION_ERROR shape and message style for consistency.
Summary
unityctl test runexecutes and reports results even when the project currently has unresolved C# compilation errors. In that state Unity runs the test suite against the last successfully compiled assemblies (stale DLLs inLibrary/ScriptAssemblies), so the command can return a green/passing result — or a result with a different test count — that does not reflect the code on disk. There is no warning that the code under test never compiled.This is a correctness/safety problem: a passing
test runafter an edit that doesn't compile is silently meaningless, and an agent or human reading the result will believe their change is verified when it was never built.Expected behavior
test runshould refuse to run (or at minimum loudly warn) when compilation has failed, consistent with howplay enterandrecord startalready behave:Proposed: return a
COMPILATION_ERRORresult (non-zero CLI exit) such asCannot run tests - compilation errors exist. Fix the errors and try again.Actual behavior
Tests run against stale assemblies and the command reports success (or a stale test list), with no indication that the current sources failed to compile.
Repro
unityctl asset refresh→ confirm a clean compile.this wont compile;to a runtime script).unityctl asset refresh→ reports the compilation error (correct).unityctl test run→ observe it still reports a passing/green result (running against the previously compiled assemblies) instead of failing on the pre-existing compile error.Root cause
HandleTestRun(UnityCtl.UnityPackage/Editor/UnityCtlClient.cs, ~L1054) builds aTestRunnerApifilter and callsExecute(...)immediately. It never checksEditorUtility.scriptCompilationFailed(orEditorApplication.isCompiling) before running.The Bridge has no precondition gate for
test.runeither — itsCommandConfig(UnityCtl.Bridge/BridgeEndpoints.cs, ~L66) only sets a timeout andCompletionEvent = TestFinished. By contrast, the play-enter path checks compilation state and short-circuits withCOMPILATION_ERROR(BridgeEndpoints.cs~L523-534), andrecord startdoes the same (~L756).EditorUtility.scriptCompilationFailedis already used elsewhere in the plugin (UnityCtlClient.cs~L1503), so the signal is readily available.Suggested fix
Guard
test runonEditorUtility.scriptCompilationFailedbefore executing — either inHandleTestRun(return an error result) or as a Bridge precondition mirroring the play-enter/record-start gate — so a stale assembly can never produce a misleading green result. Reuse the existingCOMPILATION_ERRORshape and message style for consistency.