[rezaasjd]

Summary:
AdvanceQuantizeOpAboveDefInBranchPass hoists one quantize above a def whose users are all slice/select -> quantize branches, then rewrites each original quantize as dequantize(shared) -> quantize(own qparams). The shared quantize used the qparams with the largest zero-point, without checking that they can represent the other branches.

The pattern that breaks, seen in an end-to-end model compile: one tensor feeds several slice/select branches that each have their own quantize, and the branches have very different value ranges. Some branches hold unit-range values (roughly [-1, 1.7]) and another holds values up to about 100, because the tensor packs normalized values together with large-magnitude ones. The pass hoists a quantize with the narrow qparams onto the tensor, so the wide branch reads values already clipped to [-1, 1.7] before its own quantize runs. The information is gone, and neither the graph nor the existing structural test shows it. With int8 asymmetric activations this quantized the input at about 0.1 dB SQNR and left the model outputs at 1-2 dB SQNR, one of them at -0.3 dB (noise). Skipping the hoist moved pure-quantization SQNR to 17-19 dB on two of those outputs and 43 dB on the third.

The pass now hoists only when every branch's quantize has identical qparams, in which case it behaves as before. When they differ it skips, and each branch keeps its own quantize on the original float def. Requantizing from one shared quantize either clips a branch with a wider range or coarsens one with a narrower range, so no choice of qparams is lossless for branches that differ; sharing qparams to get the hoist is a quantization decision. The old code counted qparams in quant_dict and never used the counts, then picked with sorted(..., key=lambda x: x[1]).pop(), which is the largest zero-point; that selection is removed.

This is a behavior change: hoists across differing qparams are no longer done, which costs some data volume for those values.

Downstream compile flows that use this pass see small structural changes where it no longer hoists across differing qparams: the expected node and delegate counts in five graph-structure tests change by a few nodes (one of them by a delegate). Those expectation updates are included in this diff so it passes on its own. Output numerics of the four other downstream models I measured are identical before and after.

Differential Revision: D127215700

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24132

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 71b1883 with merge base 7d9eb71 ():

NEW FAILURE - The following job has failed:

• Test QNN Windows MSVC build / build-qnn-windows-arm64 (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@rezaasjd has exported this pull request. If you are a Meta employee, you can view the originating Diff in D127215700.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.