From 3ed23be62469cbdc5df4b228c5c571a44d8ed712 Mon Sep 17 00:00:00 2001 From: David Leong <116610336+leongdl@users.noreply.github.com> Date: Sat, 1 Aug 2026 19:35:02 -0700 Subject: [PATCH 1/3] test: Add expected-failure WRAP_ACTIONS fixtures (parked in proposed/) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec-correct fixtures that FAIL against current reference implementations, parked in a proposed/ directory the conformance runner does not discover, so merging keeps the suite green. Each entry in proposed/README.md records the observed output, spec citation, and classification. Promote each fixture up one directory unchanged when its implementation fix lands. Companion to conformance-wrap-actions-gaps. Review: quorum-review fixes — kind-level proposed/ placement (jobs/proposed/); README corrected: the defect fails BOTH implementations (2026-08-12 sweep), the §5.2 ArgString conflict is now the stated promotion gate rather than a footnote (the fixture's own args are arguably spec-invalid until that is resolved), and the unit-level repr_py twin in the func-lib expected-failures PR is cross-referenced. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> --- .../WRAP_ACTIONS/jobs/proposed/README.md | 47 ++++++++++++++++ ...-escapes-newline-in-wrapped-args.test.yaml | 54 +++++++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md create mode 100644 conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml diff --git a/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md b/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md new file mode 100644 index 00000000..1c7e82e9 --- /dev/null +++ b/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md @@ -0,0 +1,47 @@ +# Proposed WRAP_ACTIONS fixtures (parked) — jobs + +Fixtures in this directory are believed spec-correct but FAIL against the +current reference implementations. They are parked here so the main suite +stays green; promotion is a mechanical move up one directory +(kind-level `//proposed/` placement). The conformance +runner does not scan `proposed/` directories. + +## wrap-repr-py-escapes-newline-in-wrapped-args + +**Classification: spec conflict to resolve first; then an implementation +bug in BOTH implementations.** + +Expression Language §2.2.6 documents `repr_py` as following Python's +`repr`, with the explicit example `repr_py("hello\nworld")` → +`'hello\\nworld'`. Both reference implementations (openjd-rs AND the +Python CLI — 2026-08-12 sweep, fails on both) instead embed the newline +raw into the emitted literal, so round-tripping a multi-line `python -c` +program through `repr_py(WrappedAction.Args)` produces broken Python: + +``` +File "", line 1 + print(repr(['-c', 'import sys + ^ +SyntaxError: EOL while scanning string literal +``` + +Reproduce: run this fixture through the conformance runner, or wrap any +action whose args contain U+000A and forward with the reference +`repr_py` pattern. + +**The promotion gate is a spec decision, not just the repr_py fix:** +Template Schemas §5.2 restricts `` to characters outside the +Cc unicode category, which excludes newlines — so this fixture's own +onRun args are arguably spec-INVALID as written, and a conforming +validator could reject the template before repr_py is ever exercised. +Neither implementation enforces §5.2 today, and several merged fixtures +(plus the common multi-line `python -c` convention, which container +queue environments also rely on) depend on the acceptance. Either §5.2 +is relaxed to permit newlines (making this fixture promotable once +repr_py is fixed), or the validator rejects them (making this fixture, +and the multi-line convention suite-wide, invalid). The two cannot both +stand; resolve the spec question first. + +Unit-level twin: `EXPR/jobs/proposed/expr2.2.6--repr-py-newline-roundtrip` +(func-lib expected-failures PR) pins the same repr_py defect without the +WRAP_ACTIONS forwarding layer; this fixture adds the end-to-end path. diff --git a/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml b/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml new file mode 100644 index 00000000..0fdb2a5c --- /dev/null +++ b/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml @@ -0,0 +1,54 @@ +# PARKED — fails against openjd-rs (implementation bug); see proposed/README.md. +# +# Expression Language §2.2 (String Functions): "repr_py follows the +# behavior of Python's repr", with the worked example +# repr_py("hello\nworld") returning 'hello\\nworld' — control characters +# in the value MUST be escaped in the emitted Python literal. RFC 0008's +# reference forwarding pattern round-trips WrappedAction.Args through +# repr_py; a wrapped action whose arg carries a real newline (a multi-line +# python -c program, the suite's own portable-fixture convention) must +# survive the round-trip and run. +# +# Against openjd-rs the newline is embedded RAW into the generated Python +# source, which dies with "SyntaxError: EOL while scanning string +# literal" — the grand-child never runs. Catches a repr implementation +# that quotes but does not escape. +template: + specificationVersion: jobtemplate-2023-09 + name: WrapReprPyEscapesNewlineInWrappedArgs + steps: + - name: Step1 + script: + actions: + onRun: + command: python + args: + - "-c" + - "import sys\nprint('GRAND_CHILD_RAN')" +environments: +- specificationVersion: environment-2023-09 + extensions: + - WRAP_ACTIONS + - EXPR + environment: + name: WrapEnv + script: + actions: + onWrapEnvEnter: + command: python + args: + - "-c" + - "import subprocess,sys; sys.exit(subprocess.run([{{repr_py(WrappedAction.Command)}}]+{{repr_py(WrappedAction.Args)}}).returncode)" + onWrapTaskRun: + command: python + args: + - "-c" + - "import subprocess,sys; sys.exit(subprocess.run([{{repr_py(WrappedAction.Command)}}]+{{repr_py(WrappedAction.Args)}}).returncode)" + onWrapEnvExit: + command: python + args: + - "-c" + - "import subprocess,sys; sys.exit(subprocess.run([{{repr_py(WrappedAction.Command)}}]+{{repr_py(WrappedAction.Args)}}).returncode)" +expected: + output: + - GRAND_CHILD_RAN From 7189e4693b166a32268f0760fbb4bf87a78b28b8 Mon Sep 17 00:00:00 2001 From: David Leong <116610336+leongdl@users.noreply.github.com> Date: Thu, 10 Sep 2026 08:35:19 -0700 Subject: [PATCH 2/3] test: Promote the repr_py fixtures out of proposed/ Both were parked as expected failures. openjd-rs#374 landed the fix, so they pass and become the regression tests the proposed/README promised. Measured against a binary built from openjd-rs main at 5b04959, and against a pre-fix binary as the control: fixture pre-fix merged EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip FAIL PASS WRAP_ACTIONS/.../wrap-repr-py-escapes-newline FAIL PASS Both pre-fix failures are "SyntaxError: unterminated string literal" with the expected output missing, which is the defect they were written for: repr_py quoted its output without escaping, so a value carrying a newline emitted it raw and CPython would not parse the result. Discovered by the runner's directory scan from their new locations, not just single-file mode, which is what leaving proposed/ was for. No regression in either category: WRAP_ACTIONS 73 passed 0 failed, EXPR 352 passed 0 failed. The EXPR fixture comes from the func-lib expected-failures branch (#165), which should drop its copy so the two do not collide on merge. Headers rewritten, since both described the parked state and claimed the implementations fail. Now spec, test, expectation in three paragraphs each, naming the paired fixture, within the comment-line budget. proposed/README.md and the proposed/ directory are removed: the fixture it described was its only entry. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> --- ...2.2.6--repr-py-newline-roundtrip.test.yaml | 36 ++++++++++++++ .../WRAP_ACTIONS/jobs/proposed/README.md | 47 ------------------- ...-escapes-newline-in-wrapped-args.test.yaml | 22 ++++----- 3 files changed, 45 insertions(+), 60 deletions(-) create mode 100644 conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml delete mode 100644 conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md rename conformance-tests/2023-09/WRAP_ACTIONS/jobs/{proposed => }/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml (57%) diff --git a/conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml new file mode 100644 index 00000000..804999b4 --- /dev/null +++ b/conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml @@ -0,0 +1,36 @@ +# Expression Language §2.2.6 defines repr_py as following Python's repr, so a +# control character in the value must be escaped in the emitted literal. +# +# repr_py("a\nb") is handed to ast.literal_eval, CPython's own parser, and the +# result compared to the input. Paired with +# WRAP_ACTIONS/jobs/wrap-repr-py-escapes-newline-in-wrapped-args, which pins the +# same rule through RFC 0008's forwarding pattern. +# +# The round-trip must print NEWLINE:PASS. A repr that quotes without escaping +# emits a raw newline, and CPython raises SyntaxError before the comparison runs. +template: + specificationVersion: jobtemplate-2023-09 + extensions: + - EXPR + name: TestJob + steps: + - name: Step1 + let: + - 'q = repr_py("a\nb")' + script: + actions: + onRun: + command: python + args: + - -c + - | + import ast + import sys + got = ast.literal_eval(sys.argv[1]) + print("NEWLINE:" + ("PASS" if got == "a\nb" else "FAIL got=" + repr(got))) + - "{{ q }}" +expected: + output: + - NEWLINE:PASS + forbidden: + - FAIL diff --git a/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md b/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md deleted file mode 100644 index 1c7e82e9..00000000 --- a/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/README.md +++ /dev/null @@ -1,47 +0,0 @@ -# Proposed WRAP_ACTIONS fixtures (parked) — jobs - -Fixtures in this directory are believed spec-correct but FAIL against the -current reference implementations. They are parked here so the main suite -stays green; promotion is a mechanical move up one directory -(kind-level `//proposed/` placement). The conformance -runner does not scan `proposed/` directories. - -## wrap-repr-py-escapes-newline-in-wrapped-args - -**Classification: spec conflict to resolve first; then an implementation -bug in BOTH implementations.** - -Expression Language §2.2.6 documents `repr_py` as following Python's -`repr`, with the explicit example `repr_py("hello\nworld")` → -`'hello\\nworld'`. Both reference implementations (openjd-rs AND the -Python CLI — 2026-08-12 sweep, fails on both) instead embed the newline -raw into the emitted literal, so round-tripping a multi-line `python -c` -program through `repr_py(WrappedAction.Args)` produces broken Python: - -``` -File "", line 1 - print(repr(['-c', 'import sys - ^ -SyntaxError: EOL while scanning string literal -``` - -Reproduce: run this fixture through the conformance runner, or wrap any -action whose args contain U+000A and forward with the reference -`repr_py` pattern. - -**The promotion gate is a spec decision, not just the repr_py fix:** -Template Schemas §5.2 restricts `` to characters outside the -Cc unicode category, which excludes newlines — so this fixture's own -onRun args are arguably spec-INVALID as written, and a conforming -validator could reject the template before repr_py is ever exercised. -Neither implementation enforces §5.2 today, and several merged fixtures -(plus the common multi-line `python -c` convention, which container -queue environments also rely on) depend on the acceptance. Either §5.2 -is relaxed to permit newlines (making this fixture promotable once -repr_py is fixed), or the validator rejects them (making this fixture, -and the multi-line convention suite-wide, invalid). The two cannot both -stand; resolve the spec question first. - -Unit-level twin: `EXPR/jobs/proposed/expr2.2.6--repr-py-newline-roundtrip` -(func-lib expected-failures PR) pins the same repr_py defect without the -WRAP_ACTIONS forwarding layer; this fixture adds the end-to-end path. diff --git a/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml b/conformance-tests/2023-09/WRAP_ACTIONS/jobs/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml similarity index 57% rename from conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml rename to conformance-tests/2023-09/WRAP_ACTIONS/jobs/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml index 0fdb2a5c..94e7b1a9 100644 --- a/conformance-tests/2023-09/WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml +++ b/conformance-tests/2023-09/WRAP_ACTIONS/jobs/wrap-repr-py-escapes-newline-in-wrapped-args.test.yaml @@ -1,18 +1,14 @@ -# PARKED — fails against openjd-rs (implementation bug); see proposed/README.md. +# Expression Language §2.2.6 defines repr_py as following Python's repr, and RFC +# 0008's reference forwarding pattern round-trips WrappedAction.Args through it. # -# Expression Language §2.2 (String Functions): "repr_py follows the -# behavior of Python's repr", with the worked example -# repr_py("hello\nworld") returning 'hello\\nworld' — control characters -# in the value MUST be escaped in the emitted Python literal. RFC 0008's -# reference forwarding pattern round-trips WrappedAction.Args through -# repr_py; a wrapped action whose arg carries a real newline (a multi-line -# python -c program, the suite's own portable-fixture convention) must -# survive the round-trip and run. +# The wrapped action's arg is a two-line python -c program, the suite's own +# portable-fixture convention. All three hooks forward it. Paired with +# EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip, which pins the same rule +# without the forwarding layer. # -# Against openjd-rs the newline is embedded RAW into the generated Python -# source, which dies with "SyntaxError: EOL while scanning string -# literal" — the grand-child never runs. Catches a repr implementation -# that quotes but does not escape. +# The grand-child must run and print GRAND_CHILD_RAN. A repr that quotes without +# escaping embeds the newline raw, and the generated program dies with +# SyntaxError before reaching subprocess.run. template: specificationVersion: jobtemplate-2023-09 name: WrapReprPyEscapesNewlineInWrappedArgs From 38555f422e49099fcfd8961a4d29ceeab430df61 Mon Sep 17 00:00:00 2001 From: David Leong <116610336+leongdl@users.noreply.github.com> Date: Thu, 10 Sep 2026 17:11:47 -0700 Subject: [PATCH 3/3] test: Assemble the repr_py round-trip FAIL marker at run time windows-latest failed the Python lane on this fixture and ubuntu-latest passed the same installed CLI. Not an implementation difference: the fixture matched its own forbidden marker. The harness scans a job's whole output for forbidden substrings, and on Windows openjd-sessions-for-python echoes the full child command line at INFO (_subprocess.py logs list2cmdline(self._args)); the POSIX path logs only the temp .sh path. This fixture's script source contained the literal FAIL, and FAIL is its forbidden entry, so the echo matched even though every assertion passed. The run log discriminates the two cases. It reports "Found forbidden output: FAIL" with no "Missing expected output" line, so NEWLINE:PASS was printed and the escaping worked on Windows too. A genuine failure of this fixture looks different: the pre-fix control run reported the missing NEWLINE:PASS line alongside a SyntaxError. Split the marker across two adjacent literals, the convention this repo already uses for the same hazard (20c1a84, and the timeout fixture's 'SHOULD_NOT' + '_PRINT'). Python concatenates them at compile time, so the run-time value is unchanged at 'FAIL got=' while the source no longer carries the literal. Verified both halves: the value still compares equal, and the script body no longer contains FAIL. Verified the marker still catches a real failure: comparing against a wrong value makes the runner report 0 passed, 1 failed, and restoring the fixture byte-identically returns it to 1 passed, 0 failed. The Rust lane was never affected, because it prints only COMMAND_OUTPUT records while the command echo is tagged FILE_PATH|PROCESS_CONTROL. The WRAP_ACTIONS fixture is unaffected too: it declares no forbidden entries. Windows itself is unverified from here. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> --- .../EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml b/conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml index 804999b4..8fb5d2d1 100644 --- a/conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml +++ b/conformance-tests/2023-09/EXPR/jobs/expr2.2.6--repr-py-newline-roundtrip.test.yaml @@ -8,6 +8,8 @@ # # The round-trip must print NEWLINE:PASS. A repr that quotes without escaping # emits a raw newline, and CPython raises SyntaxError before the comparison runs. +# The failure marker is split across two literals because Windows echoes the +# child command line into the output the harness scans for forbidden strings. template: specificationVersion: jobtemplate-2023-09 extensions: @@ -27,7 +29,7 @@ template: import ast import sys got = ast.literal_eval(sys.argv[1]) - print("NEWLINE:" + ("PASS" if got == "a\nb" else "FAIL got=" + repr(got))) + print("NEWLINE:" + ("PASS" if got == "a\nb" else "FA" "IL got=" + repr(got))) - "{{ q }}" expected: output: