Skip to content

[PyTorch][torch.compile] Differentiable fp8_output and fp8_grad for Linear - #47

Open
pggPL wants to merge 2 commits into
mainfrom
linear_compile_fp8_io
Open

[PyTorch][torch.compile] Differentiable fp8_output and fp8_grad for Linear#47
pggPL wants to merge 2 commits into
mainfrom
linear_compile_fp8_io

Conversation

@pggPL

@pggPL pggPL commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Description

te.Linear under torch.compile currently falls back to eager for fp8_output=True with gradients enabled and for fp8_grad=True when the input needs a gradient (compile_unsupported_reason: "differentiable fp8_output=True", "a quantized input grad (fp8_grad=True)"). This PR makes both work on the compiled custom-op path.

Why it did not work:

  • The op returns a quantized output as flat inner buffers (uint8 data, scales) and the Float8Tensor wrapper was rebuilt outside the op as a leaf, so autograd had no path from the wrapper back to the op.
  • AOTAutograd guesses the tangent of a subclass output from the output's own metadata (E4M3, forward quantizer). The real gradient coming from an fp8_grad=True consumer is E5M2 with a different quantizer, which fails with "guessed its metadata incorrectly" unless the subclass implements the coercion hooks.
  • A quantized dgrad was returned from the backward op as a subclass smuggled inside a Tensor[] slot; this worked by accident in default mode and broke CUDA graphs (buffers not tracked as outputs).
  • Float8Tensor.dequantize() called a raw tex.dequantize binding, so it could not be traced inside a graph (including the backward of any torch op receiving a Float8 tangent).
  • QuantizedTensor.__torch_dispatch__ re-dispatched through super().__torch_dispatch__, which disables Python dispatch and bypasses the inner FakeTensor/FunctionalTensor under AOTAutograd (a real CUDA kernel was launched on a fake tensor: illegal memory access).

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • dynamo/custom_op.py: a quantized user output carries one extra "grad handle" slot (stride-0 tensor of the output's logical shape and dtype). _QuantizedOutputFn rebuilds the wrapper from the inner buffers and routes the wrapper's gradient (plain or quantized) to the handle; _slice_user_grads reads the handle's grad. Quantized grads returned by the backward op are flattened to inner buffers and rebuilt in _autograd_backward from the bwd fake specs (_unpack_bwd_result).
  • quantized_tensor.py: __coerce_tangent_metadata__ (tangent template from Quantizer.tangent_quantizer(): rowwise-only, E5M2 for Float8 quantizers) and __coerce_same_metadata_as_tangent__ (same type/dtype: fix usage and pass through; different FP8 dtype: requantize with the expected quantizer; expected plain tensor: dequantize). __torch_dispatch__ re-dispatches with func(*args, **kwargs) after dequantizing.
  • float8_tensor_storage.py: FP8 dequantize goes through a torch.library custom op with a fake kernel, so it traces under torch.compile.
  • module/linear.py: LinearBwdArgs.grad_output and LinearFwdArgs.inp are TensorOrQuantized, so a quantized dgrad and a Float8Tensor input cross the op boundary through the wrapper op's subclass flattening; the fp8_output / fp8_grad / quantized-input compile gates are removed.
  • Tests (test_torch_compile.py): Linear(fp8_output=True) feeding a second Linear inside one graph (consumer dgrad in bf16 or FP8), an externally quantized input; fp8_grad with the dgrad consumed in-graph (TE Linear or torch op) and crossing the graph boundary; differentiable fp8_output with in-graph dequantize(), with an E5M2 tangent, with an E4M3 tangent (requantization), and a plain output receiving a Float8 tangent. All bit-exact against eager except the requantization case.

Known PyTorch limitations, pinned by tests (the first one also means a graph break right after a differentiable fp8_output=True Linear turns the Float8Tensor into a graph output and fails in backward unless the consumer produces a quantized grad; on main that configuration ran eagerly):

  • A plain-tensor gradient for a quantized graph output (e.g. .dequantize() outside the graph) raises AOTAutograd's tangent-metadata error; there is no coercion hook for plain -> subclass.
  • A quantized dgrad for a plain graph input is only re-wrapped when the graph has a tensor-subclass input or output (requires_subclass_dispatch); otherwise it is lifted as a fake constant (xfail).

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

…inear

Quantized op outputs carry a grad-handle slot; _QuantizedOutputFn rebuilds
the wrapper so autograd reaches the op through the handle. Quantized grads
are flattened across the backward op boundary and rebuilt from the bwd fake
specs. QuantizedTensor gets the AOTAutograd tangent coercion hooks, FP8
dequantize becomes a traceable custom op, and QuantizedTensor dispatch
re-dispatches instead of disabling torch dispatch.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Declare inp as TensorOrQuantized so a Float8Tensor input is flattened by the
wrapper op like the other quantized operands, and drop the quantized-input
fallback. This lets Linear(fp8_output=True) feed another Linear inside one
graph; the consumer's dgrad (plain or quantized) reaches the producer through
the grad handle.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant