Fall back instead of baking a symbolic dim into softplus and nll_loss - #2812
Open
LeSingh1 wants to merge 1 commit into
Open
Fall back instead of baking a symbolic dim into softplus and nll_loss#2812LeSingh1 wants to merge 1 commit into
LeSingh1 wants to merge 1 commit into
Conversation
Both converters read a shape dim and use it where only a constant works, so a flexible input shape aborts the conversion. softplus with beta != 1 on a rank 4 input builds per channel alpha and beta for softplus_parametric out of x.shape[1]. A dynamic channel dim raises "TypeError: Cannot convert symbols to int". Take the existing general decomposition, softplus(beta * x) / beta, in that case; it is what every other rank already uses and is numerically the same. nll_loss with reduction="mean" divided by np.float32(x.shape[0]) before summing, which raises "TypeError: Cannot convert expression to float" on a dynamic batch. reduce_mean over the batch axis is the same value and needs no constant, so the batch size is no longer read at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Same class as #2808: both converters read a shape dim and use it where only a constant works, so a flexible input shape aborts the conversion.
softpluswithbeta != 1on a rank 4 input builds per-channelalpha/betaforsoftplus_parametricout ofx.shape[1]:Falls back to the general decomposition
softplus(beta * x) / betawhen the channel dim is dynamic — the same path every other rank already takes, and numerically identical.nll_losswithreduction="mean"divided bynp.float32(x.shape[0])before summing, which raisesTypeError: Cannot convert expression to floaton a dynamic batch.reduce_meanover the batch axis is the same value and needs no constant, so the batch size is no longer read at all."none"and"sum"never used it and are unchanged.Testing
test_softplus_beta_dynamic_channelandtest_nllloss_dynamic_batch(all three reductions). 4 of the 8 cases fail onmain— the"none"and"sum"reductions pass either way and are there as controls. Existingtest_softplusandtest_nlllossare unchanged and still pass. Checkednll_lossnumerically at a batch size different from the traced one.This machine has a broken scikit-learn install that makes every
TorchFrontend.TORCHEXPORTcase error out, including pre-existing ones, so I could only exercise the TorchScript frontend locally.