Skip to content

[STUBGEN][RUST] Generate complete object mirrors and allocators - #740

Merged
tlopex merged 8 commits into
apache:mainfrom
Seven-Streams:main-dev/2026-09-03/stubgen_rust_complete
Sep 4, 2026
Merged

[STUBGEN][RUST] Generate complete object mirrors and allocators#740
tlopex merged 8 commits into
apache:mainfrom
Seven-Streams:main-dev/2026-09-03/stubgen_rust_complete

Conversation

@Seven-Streams

@Seven-Streams Seven-Streams commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Attach the layout classifier (#730) to the Rust backend. A type whose layout the registry proves is now bound complete: a #[repr(C)] struct with every physical field at its reflected offset and width, a const size/alignment assertion, and a lossless allocator (<Leaf>Obj::new crate-private, <Leaf>::new public). Everything else keeps the opaque form of #738. Three new directives: opaque vetoes a reproducible layout, upcast adds a hand-written typed view, custom-new renames the generated allocator to from_complete_fields so a hand-written new can own the name.

Motivation

This is what tvm-rust-ext's STUBGEN_FEEDBACK.md asks for: ordinary data nodes allocated in Rust from their complete fields, polymorphic and unreflected-byte types kept opaque. Generating ir. / tirx. / arith. / target. from tvm-rust-ext's libtvm_compiler.so gives 131 types (98 complete, 33 opaque) that compile against the crate without edits; the differences from the hand-written bindings are exactly what the directives cover.

Changes

  • stub/layout.py: classify(..., unmirrored=...) and the no-mirror reason, so types under a builtin parent (ffi.Enum, ...) stay opaque: their base is only a header-only stand-in.
  • rust_generator/codegen.py: classify each object with its ancestors; mirror fields (scalars by reflected width, Optional<T> as Option<T> or tvm_ffi::Optional<T> by payload, directive widths checked against the field size); render the complete struct and its allocators; upcast and custom-new handling.
  • rust_generator/directives.py, consts.py: the three directives and the width tables.
  • rust_generator/utils.py: a reflected field named base or data is spelled base_ / data_, since those are the generated struct's own members (TVM has tirx.Ramp.base, tirx.DeclBuffer.data).
  • examples/rust_stubgen/: IntPair is now a plain data object, allocated from Rust with the generated IntPair::new and read back by C++.

Testing

test_stubgen_rust.py (36 cases), test_stubgen.py, test_stub_layout.py: 121 passed; ruff clean. The example regenerates an identical mod.rs and cargo run prints a=1 b=2 kind=PairKind(1) and sum=3.

@Seven-Streams Seven-Streams changed the title Main dev/2026 09 03/stubgen rust complete [STUBGEN][RUST] Generate complete object mirrors and allocators Sep 4, 2026
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
A type under a builtin such as `ffi.Enum` embeds a header-only stand-in
for it, but the classifier judged the builtin from its registry bytes and
called a fieldless `testing.TestEnumVariant` complete: the generated
struct then asserted `size_of == 48` on 24 bytes and grew allocators.

`layout.classify` gains an `unmirrored` set and a `no-mirror` reason for
types whose bytes the target never reproduces; the Rust backend passes
every builtin ancestor below `ffi.Object`, so such types and everything
under them stay opaque with an explanatory verdict.

Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
@Seven-Streams
Seven-Streams force-pushed the main-dev/2026-09-03/stubgen_rust_complete branch from a6229b6 to ef00323 Compare September 4, 2026 19:33
@Seven-Streams
Seven-Streams marked this pull request as ready for review September 4, 2026 19:39
@tlopex
tlopex merged commit 24cfa44 into apache:main Sep 4, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants