Skip to content

native: lower bounded Plan 9 assembly to naked LLVM functions - #42

Merged
xushiwei merged 6 commits into
xgo-dev:mainfrom
zhouguangyuan0718:codex/native-arm64-source-20260923
Sep 24, 2026
Merged

xushiwei merged 6 commits into
xgo-dev:mainfrom
zhouguangyuan0718:codex/native-arm64-source-20260923

Conversation

@zhouguangyuan0718

@zhouguangyuan0718 zhouguangyuan0718 commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Raw foreign callbacks and register-shuffling stubs have physical native interfaces without Go function declarations. Typed lowering cannot infer their signatures, and the native path introduced in #41 depended on Go assembler object internals.

Replace that object path with bounded source translation and TranslateNativeModule. Each local TEXT becomes an internal naked noinline LLVM function containing one side-effecting inline-asm call and unreachable. DATA/GLOBL becomes packed LLVM global definitions with real pointer initializers. The caller owns the returned module. LLGo serializes it to textual .ll and reuses the ordinary Plan 9 assembly compiler, object/archive and LTO path, bypassing signature-based ABI rewrites for native carriers. Other consumers may merge it into a larger LLVM module. No module asm, Go object reader, Go assembler, or separate native assembly step is needed.

The carrier's void() type is not an inferred Go/C signature. Entry arguments, results, stack alignment and callee-saved registers remain the assembly's physical-register contract; consumers retain their own actual calling ABI. LLVM 22 explicitly preserves mismatched naked prototypes in InstCombineCalls. No typed IR call to an unknown callee is synthesized, and the two unsupported-signature void() fallbacks in typed lowering remain removed. Returning carriers are never marked noreturn; implicit fallthrough between TEXT functions is rejected.

Symbol references are constant LLVM inline-asm operands (s on AMD64 PIC, i on ARM64), and local branch labels use unique asm IDs. This preserves references through LLVM module renaming and dead-code elimination. Imports are untyped external byte addresses, with no invented function prototype. Native CALL/JMP stays inside assembly.

Supported pairs are Linux and Darwin on AMD64 and ARM64. Shared source validation, directive handling and instruction lowering also retain the standalone TranslateNativeSource API. Unsupported frames, instructions, preprocessing, references and addressing fail explicitly. The precise supported subset and naked-carrier contract are documented in doc/native-arm64.md. LLGo #2623 consumes the module API.

Validation with LLVM 22:

  • Root and both command-module test suites pass.
  • All four target pairs cross-assemble. C ABI tests execute on Darwin/ARM64 and in Linux/amd64 + Linux/arm64 containers; Darwin/amd64 remains cross-object only.
  • O2/full-LTO optimization tests place actual parameterized, integer/floating-result calls and void() carriers in the same module. They cover external calls, argument shuffles, return values, explicit frames and local control flow.
  • Module merge tests rename identically named local functions/labels, retain assembly-referenced callees, remove unreachable carriers, and resolve an external assembly address to a typed LLVM function definition. DATA tests cover forward references, relocations, gaps, truncation and mutable storage.
  • LLGo integration passes actual off/thin/full LTO builds and execution on Darwin/ARM64 with matching LLVM/LLD 22. Cross-driver tests validate the shared .ll compiler path, saved IR, separate ELF/Mach-O emission and DATA size-mismatch rejection, without a usable GOROOT.
  • Full forced Darwin/ARM64 TypeScript Go tsc build at c975de5011fb7dfb32a491cf3fcf02d4f811f50e passes. Version, strict ES2022 JS/declarations, cross-file imports, TS2322 diagnostics/exit status and emitted JS execution match standard Go.

Assembly instructions remain opaque to IR optimization. Go ABI adapters, runtime transitions, stack maps and unwinding are outside this contract. Full Linux tsc builds and watch-mode correctness are not established by this validation.

@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.28358% with 45 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
native_arm64_source.go 88.94% 21 Missing ⚠️
native_source.go 95.17% 11 Missing ⚠️
native_amd64_source.go 94.30% 7 Missing ⚠️
native_module.go 95.12% 6 Missing ⚠️

📢 Thoughts on this report? Let us know!

fennoai[bot]

This comment was marked as outdated.

@zhouguangyuan0718 zhouguangyuan0718 changed the title arm64: translate foreign stubs directly to native assembly native: translate bounded Plan 9 source for Linux and Darwin Sep 23, 2026
@zhouguangyuan0718 zhouguangyuan0718 changed the title native: translate bounded Plan 9 source for Linux and Darwin native: lower bounded Plan 9 assembly to naked LLVM functions Sep 23, 2026
@zhouguangyuan0718

Copy link
Copy Markdown
Contributor Author

/review Please review the current head 66e8f32. The earlier review predates the Linux/amd64 + Linux/arm64 generalization and naked LLVM function backend. Focus on semantic correctness, physical-register arguments/results, inline-asm symbol operands and relocations, DATA layout, optimization/LTO, and explicit rejection of unsupported Plan 9 forms. Please report actionable findings against this head; this request is for review only, not code changes.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: native physical-register backend (head 66e8f32)

I re-reviewed the current head against the four requested areas. The design is defensively layered and semantically careful: every value that reaches emitted output flows through a validator — the nativeName regex (^[A-Za-z_][A-Za-z_0-9]*$), the integer-only expression evaluator (nativeInteger/parseImmExpr, no identifiers), a fixed register/opcode map, or a compiler-generated label. I could not construct an operand that smuggles raw text into either the emitted .s or the LLVM inline-asm template. I confirmed the items you called out are correct:

  • Physical-register args/results: the naked void() carrier (native_module.go) is address-only; carrier ends with a RET/JMP/B check, naked noinline, no noreturn, ~{memory} clobber — call sites keep their own ABI.
  • Inline-asm symbol operands & relocations: symbol operands use s (amd64 PIC) / i (arm64); $→$$ escaping is applied before ${:uid}/${N:c} template insertion, in the correct order. amd64 imported calls get @PLT only on ELF; arm64 bl/b produce the assembler's native branch relocations.
  • DATA layout: offset/width/overlap/bounds checks are sound; g.Size bounded to (0, 64MiB]; address DATA fixed at width 8; forward references handled via a packed LLVM struct.
  • Explicit rejection of unsupported forms: the nativeSource sanitizer rejects quotes/backslashes/block-comments/non-textflag.h directives; SP/FP/g/R18/R31/W-aliases rejected on arm64; register-with-SP arith rejected; undeclared file-local TEXT no longer receives an invented void() signature (go_translate.go).
  • Docs: doc/native-arm64.md and README claims were checked line-by-line against the implementation and all match.

One actionable correctness gap (inline): DATA integer values are not range-checked against their declared width and are silently truncated by both emitters. See the inline comment.

Minor / maintainability (no inline)

  • dataValues() recomputed per call: it re-scans and re-sorts f.Data for every global, and TranslateNativeModule computes the identical result 3× per global (native_source.go:193, native_module.go:56 & :104); the emit-time calls also discard the error, relying on the earlier validation pass. Correct today, but caching the grouped/sorted slice once in prepareNativeSource would remove the redundancy and the fragile silent error-drop.
  • Shared mutable state on nativeEmitter (native_source.go:300-304): the symbolOperand closure is only valid on the module path and must be nil on the source path, and e.labels is rewritten in place per function. A short doc comment on which backend owns each field, or splitting per-function state out, would reduce misuse risk.
  • LABEL emit uses a bare map lookup (native_amd64_source.go:93, native_arm64_source.go:96): e.labels[a[0].Sym] emits an empty : label on a map miss. Unreachable in practice (functionLabels pre-populates and validates), but a latent hazard if the collection and emit passes ever diverge; an explicit presence check would be safer.

No blocking issues; this is review-only feedback as requested.

Comment thread native_source.go
if d.Width < 8 {
mask = (uint64(1) << (8 * d.Width)) - 1
}
fmt.Fprintf(&out, "%s %#x\n", directive, d.Value&mask)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] DATA integer values silently truncated to declared width

Silent truncation of over-wide DATA values. Neither nativeSource (which only checks the initializer parses as an integer) nor dataValues (which checks width is 1/2/4/8 and bounds) verifies that d.Value fits its declared width. Here the value is masked to the width, and native_module.go:118 (llvm.ConstInt(ctx.IntType(int(d.Width*8)), d.Value, false)) truncates identically. So DATA ·p(SB)/1, $0x1234 silently emits .byte 0x34 instead of erroring. cmd/asm rejects this as "value out of range". This is the class of silent-wrong-code the backend otherwise avoids. Suggest rejecting any non-address value whose magnitude does not fit the width (accepting both signed and unsigned representations).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the concrete example with native Go 1.27.0: DATA ·p(SB)/1, $0x1234 is accepted on darwin/arm64, linux/amd64 and linux/arm64. A native Darwin executable reading the byte prints 52 (0x34). cmd/asm asmData calls LSym.WriteInt, which explicitly writes byte(i), uint16(i), or uint32(i), truncating to the declared width: https://github.com/golang/go/blob/go1.27.0/src/cmd/internal/obj/data.go#L100 . Thus the current source and LLVM emitters match Go semantics; rejecting this would be a compatibility regression. TestNativeNakedDataAndIRReferences already covers /1 with $0x122 and checks the resulting byte through executable harnesses. No range check added.

@xushiwei
xushiwei merged commit 7cc8c0f into xgo-dev:main Sep 24, 2026
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants