Fix 8-bit fixed right shifts and rotations of signed integers - #1421
Open
Arthur031221 wants to merge 1 commit into
Open
Arthur031221 wants to merge 1 commit into
Arthur031221 wants to merge 1 commit into
Conversation
The unsigned 8-bit bitwise_rshift<N> on SSE2 and AVX2 masked with the left shift mask (1 << N) - 1 instead of 0xFF >> N, the signed 8-bit version on SSE2 built a wrong sign mask, and the AVX2 version returned zero for N == 0. Rotations were written with arithmetic right shifts, so rotl/rotr of a negative value filled the vacated bits with the sign. Rotate through the unsigned type in the common kernels and in the scalar overloads, and accept a rotation count of 0.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anyone calling
bitwise_rshift<N>onbatch<uint8_t>orbatch<int8_t>, orrotl/rotron a signed batch or scalar, got wrong lane values.On
batch<uint8_t>with-msse2or-mavx2,bitwise_rshift<1>of a batch filled with0x81returned0x00instead of0x40in every lane, andbitwise_rshift<7>returned3instead of1on SSE2. The mask applied after the 16-bit shift was(1 << N) - 1, the mask for a left shift, instead of0xFF >> N. On AVX2bitwise_rshift<0>of auint8_tbatch returned zeros. The signed 8-bit version on SSE2 (and the AVX512VL 128-bit kernels that inherit it) also returned wrong values for some lanes (I checked every 8-bit input against the scalar result).rotlandrotrwere written as(x << n) | (x >> (bits - n))on the signed type, so the right shift is arithmetic and a negative value gets sign bits instead of the bits shifted out. Forint32_tbatches on SSE2,rotl(INT_MIN, 1)returned-1instead of1. Native AVX512 rotate instructions were already right for 32 and 64 bit lanes, but the 8 and 16 bit lanes and the AVX/AVX2/SSE paths all went through that formula.rotl<0>androtr<0>did not compile because of thestatic_assertonbits - count.The fix:
0xFF >> Nas the byte mask in the SSE2 and AVX2 unsigned kernels,(x ^ m) - m,xsimd_scalar.hpp, and let a count of 0 return the input.The two fixes are in one change because the compile-time
rotl<N>/rotr<N>of an 8-bit batch are built onbitwise_rshift<N>, so the 8-bit rotation tests only pass with both. The scalarrotl/rotrnow reduce the count modulo the bit width, which replaces the undefined shift by the full width that a count of 0 used to produce.Tests:
test_xsimd_api.cppgets one case that comparesbitwise_rshift<0..7>of every 8-bit value against the scalar result, and one that rotates the sign bit throughrotl/rotr(runtime and compile-time counts, batch and scalar types). The existing rotation tests only used the value 12, and compared against the same formula the implementation used.Results on an AVX-512 machine with g++,
test_xsimd_api, built three times (-march=native,-mavx2,-msse2): 165 test cases pass in each. With the source changes reverted and the tests kept, 11, 12 and 12 test cases fail respectively.test_batch_int,test_bitandtest_batch_manipstill pass with-mavx2. Only x86 was run; NEON has its own fixed shift kernels and the other backends reach the common kernels, which I checked by reading only.