TilemapGPULayer: Fixing display issue on some Android devices (S23, ZFold 7, etc...)

[moufmouf]

This PR (delete as applicable)

• Fixes a bug

Bug description:

We recently started using the new TilemapGPULayer in production. While this works great on most computers, some users declared the TilemapGPULayer are not displayed at all on some Android devices (S23, S23 Ultra, ZFold 7, etc...), while still working on other Android devices (Pixel 9 Pro).

Here is a display on the issue on a S23 phone:

We should not see the "Start" layer because it is hidden by a floor layer that is implemented with TilemapGPULayer and not visible at all.

Why this is happening only on some phones?

The issue comes from a rounding error that is not treated with the same precision depending on the phones.

TilemapGPULayer.frag declares its layer data sampler with no precision qualifier:

uniform sampler2D uLayerSampler;

uLayerSampler contains pure packed 32-bit integer record data (the tile number for each tile in the RGB part) and whether the tile is flipped / rotated / empty in alpha.

But sampler2D does not store integers. It's designed to store pixel colors, essentially as a float or double value.

On a driver that implements lowp literally, the fetched alpha comes back a fraction below
its true value, so texel.a * 255.0 lands just under 16.0 and the bit-exact empty tile
test never matches:

/* Check for empty tile flag in bit 28. */
if (flags == 16.0)

tile.empty is then false for every cell in the layer. getTileTexelCoord() returns
vec2(0.0) for what it still believes is a real tile, so each empty cell samples tile 0
of the tileset — opaque in most tilesets — and paints it over everything drawn beneath
that layer. On a map with several stacked GPU layers sharing a tileset, the topmost one
wins and the map disappears under a grid of whatever tile 0 happens to be.

Reproduction

Standalone WebGL, no Phaser: upload a 1×1 RGBA8 texture with alpha byte 16 (exactly what
generateLayerDataTexture() writes for an empty tile), read it back through a fragment
sampler, and run texel.a * 255.0 == 16.0. The only difference between the two rows is the
sampler's precision qualifier.

Galaxy S23 — ANGLE (Qualcomm, Adreno (TM) 740, OpenGL ES 3.2), Chrome 150:

sampler Header	flags == 16.0	decoded alpha
default (lowp)	false	15.996
highp	true	16.004

Desktop — ANGLE (Intel, Mesa Intel(R) UHD Graphics (CML GT2)), Chrome 150:

sampler Header	flags == 16.0	decoded alpha
default (lowp)	false	16.004
highp	true	16.004

Confirmed broken on Adreno 740/750 (Galaxy S23, S23 Ultra, Z Fold 7) and fine on desktop,
Mali (Pixel 9) and older Adreno/Xclipse (Galaxy S22) — those promote lowp to fp16/fp32,
where 16/255 * 255 rounds straight back to 16.0 and the bug is invisible. I'm happy to
attach the repro page if useful.

The fix

1. Declare uLayerSampler and uAnimSampler through a DATA_PRECISION macro defined
   inside the existing GL_FRAGMENT_PRECISION_HIGH guard, so the shader still compiles
   where highp is unavailable in fragment shaders. This is the load-bearing change.
   uMainSampler is deliberately left at the default: it samples the tileset image as
   colour, where the default precision is correct and cheaper on mobile.
2. Add texelBytes(), which snaps a fetched UNORM8 texel back to exact 0–255 values before
   anything decodes it, and route getLayerData() and floatTexel() through it. The
   channels there are weighted by powers of 256, so a sub-byte fetch error becomes an error
   of up to 65536 in the decoded tile index. The reporting device's drift was small enough
   to spare the index and break only the exact comparison, but nothing bounds it that
   tightly. This also makes flags == 16.0 exact rather than merely very likely to be
   exact, which is why I left that comparison as-is.

Worth flagging one trap: changing == 16.0 to a threshold or bit test is not a fix on
its own. The observed value is 15.996, which fails >= 16.0 just as it fails == 16.0.
Raising the sampler precision fixes it, rounding fixes it; doing both makes the decode
robust regardless of how a driver interprets sampler precision.