A richly textured chapel rendered entirely in software on an ESP32-S3, using Jet. A 90-second camera tour explores the interior, stained glass and coloured light baked into the surrounding stonework; an optional controller lets you look around freely.
This is an ESP32 adaptation of James D. Lambert's N64 megatexture demo. The chapel, its UV layout and baked artwork come from that project. James' use of tiled textures on constrained hardware inspired this version. His MIT licence is preserved alongside source attribution and conversion details.
The water-gun window is especially striking when looking slightly down: its colours spill across the recessed stone sill, with the carved trim below it. These lighting effects are part of James' baked artwork, carried through the paletted textures rather than calculated as dynamic lighting.
These are native software captures of the ESP32 visual configuration, with the FPS overlay hidden: 480×320 output, RGB565, half-width rendering and alternating fields. They are not photographs of the LCD or higher-resolution desktop renders. The GIF is a short selection of tour passages; its playback rate is not a hardware benchmark. Capture details and regeneration.
- Baked lighting on simple geometry: 325 triangles across 108 surfaces, with the visual detail carried by 18 textured materials.
- Paletted mipmapped textures: a 5.01 MiB texture pack fits in PSRAM, while 256-colour RGB565 palettes stay in faster internal RAM. Perceptually weighted palette selection and error diffusion preserve bright colours and subtle lighting.
- Perspective-correct texture mapping: adaptive short spans reduce the cost of perspective division while limiting texture-coordinate error.
- A moving cache of bilinear results: frequently sampled UV regions are filtered once, then fetched as RGB565 pixels. Cold or evicted regions use nearest sampling immediately and become smooth as their tiles are ready.
- ESP32 display scheduling: two raster workers, DMA scanout and field pacing, sharing the runtime from JetExamples.
The full reduced source pack is resident in PSRAM in the default configuration. This is not yet an out-of-core demo of a scene larger than RAM. The original source-tile cache remains available for experiments, but is disabled because it did not materially improve this scene's measured performance. The separate prefiltered hot cache is enabled and is responsible for the smoothing speedup.
Fitting this artwork into 8-bit indices needs more care than simply keeping the most common colours. Noise and near-duplicate shades in the source can consume palette entries that would do more for the stained glass, beach-ball colours and soft coloured light on the stone. Our offline converter combines:
- Perceptual palette fitting in Oklab, with extra weight on lightness error to protect the shading that gives the room its depth.
- Extra priority for bright, saturated colours, so small colourful features are represented even among large areas of muted stone.
- Edge-aware cleanup of palette training data, reducing the influence of low-contrast colour speckle. The source pixels used for quantization retain their original detail; this cleanup only guides palette selection.
- Optimization against the actual RGB565 colours the display can show, merging duplicates and refilling their slots during fitting. All 18 current palettes use 256 distinct displayable colours.
- Floyd–Steinberg error diffusion over each complete mip before tiling, distributing the remaining quantization error without restarting at every 32×32 tile boundary. Each mip comes from the undithered RGB source image.
The perceptual model guides palette selection; diffusion is Pillow's standard Floyd–Steinberg implementation. This is lossy conversion, not a claim to recover the original colours exactly. It produces cleaner colours and less conspicuous dithering in this scene without adding any per-frame work or enlarging the pack.
Palettes are per source texture/material, not per tile: each material shares one 256-entry RGB565 palette across all its tiles and mip levels. Eighteen palettes occupy 9 KiB of internal RAM; the complete indexed/mipmapped pack remains 5,253,888 bytes. The approved pack is reproducible with the pinned conversion dependencies. Conversion measurements record the chosen Oklab error metrics; they are diagnostics, not universal quality scores.
Use ESP-IDF 6.0.x (tested with 6.0.1), Git, and an ESP32-S3 with 16 MiB flash and 8 MiB PSRAM. Open an ESP-IDF terminal with its compiler and Python environment active. Review the board settings below before flashing.
git clone --recurse-submodules https://github.com/CubeCoders/JetMegatexturesDemo.git
cd JetMegatexturesDemo/esp32-streaming-chapel
idf.py -B build-s3 "-DIDF_TARGET=esp32s3" "-DSDKCONFIG=sdkconfig.s3" build
idf.py -B build-s3 "-DIDF_TARGET=esp32s3" "-DSDKCONFIG=sdkconfig.s3" -p PORT flash monitorReplace PORT with your serial port, such as COM6 or /dev/ttyACM0.
Exit the monitor with Ctrl+]. The automatic tour starts without buttons.
Generated assets are checked in: ordinary builds do not need Blender, Pillow,
NumPy, Git LFS or the original N64 build tools.
If you cloned without dependencies, run git submodule update --init --recursive
from the repository root. Use the pinned revisions: Jet includes the tiled
sampler and span support this project needs, and LovyanGFX uses CubeCoders' DMA
optimizations. Keep components, cmake and the IDF project in this layout.
Test hardware: dual-core ESP32-S3 at 240 MHz, 8 MiB octal PSRAM at 80 MHz, 16 MiB flash, and a 320×480 ST7796 SPI LCD at 80 MHz, rotated to 480×320 landscape. This demo's hardware validation is for S3; the shared runtime contains P4 code but this project does not ship a tested P4 configuration.
Change the LovyanGFX setup in
components/esp32_jet/Board.hpp, rather than
editing the display library itself.
| Setting | Where to change it |
|---|---|
| SCLK, MOSI, D/C, CS, reset, backlight | Board::clock, mosi, dc, cs, reset, backlight |
| SPI frequency and mode | b.freq_write, b.freq_read, Board::spiMode |
| Display controller | lgfx::Panel_ST7796; choose your panel's LovyanGFX class |
| Dimensions, offsets, inversion, RGB/BGR | The panel.config() block |
| Backlight polarity/PWM | The light.config() block |
| Landscape orientation | tft.setRotation(3) in components/esp32_jet/Display.cpp |
| Output resolution | SCREEN_WIDTH/HEIGHT in Display.hpp, plus scene/projection assumptions |
| CPU clock, flash size, PSRAM type/speed | sdkconfig.defaults*, or idf.py ... menuconfig |
Reference S3 wiring is SCLK 46, MOSI 3, D/C 8, CS 17, reset 18, backlight 9, with no MISO and SPI mode 1. These pins are specific to our board. Connect power and ground to suit your display module. The fast S3 scanout owns a dedicated SPI2 bus and its DMA completion interrupt; do not share that bus with touch, an SD card or another display task. Another SPI peripheral or a parallel/RGB/DSI panel requires adapting scanout as well as the panel setup.
Slower displays will not deliver the same performance. At this resolution, 60 alternating fields/s require almost 74 Mbit/s of pixel data before commands; a 40 MHz SPI link cannot sustain that cadence. Select a clock your LCD and wiring actually support. The scene, viewport and buffers assume 480×320, so changing the panel dimensions alone is not enough to change resolution.
idf.py -B build-s3 "-DIDF_TARGET=esp32s3" "-DSDKCONFIG=sdkconfig.s3" menuconfigGenerated sdkconfig.s3 overrides defaults; editing sdkconfig.defaults* after
configuration will not replace existing settings. The supplied single-app
partition layout uses the full 16 MiB flash and has no OTA slot. An 8 MiB
flash board needs a smaller partition layout and the correct flash setting;
boards with less PSRAM need a different residency/cache budget. Those variants
have not been hardware-tested. Firmware is approximately 5.4 MiB.
The tour works without any input hardware. CHAPEL_BUTTONS defaults to OFF,
so the demo does not drive the reference controller pins. Enable it only for the
matching shift-register circuit, or adapt
main/BoardInput.hpp to your controls:
idf.py -B build-s3 "-DIDF_TARGET=esp32s3" "-DSDKCONFIG=sdkconfig.s3" -DCHAPEL_BUTTONS=ON buildThe reference scan uses latch GPIO20, clock GPIO19, data output GPIO47 and button input GPIO21 with pulldown, scanning eight one-hot positions. These are not direct GPIO button inputs. The first seven bits map to Left, Forward, Backward, Right, Fast, Look and Height; the eighth is unused.
| Input | Action |
|---|---|
| D-pad up/down | Forward/backward |
| D-pad left/right | Turn |
| Hold Look + up/down | Look up/down |
| Hold Look + left/right | Strafe |
| Hold Height + up/down | Rise/lower |
| Hold Height + left/right | Strafe |
| Hold Fast | Faster movement |
| Forward + backward together | Return to the automatic tour |
Any input takes over at the current camera pose. The return chord blends back into the paused tour. This is an inspection camera with outer bounds, not player physics or collision against every furnishing.
Normal settings are CHAPEL_HOT_FILTER=ON, CHAPEL_PSRAM_BACKING=ON,
CHAPEL_USE_CACHE=OFF, with all benchmark flags off. Painter groups from the
source scene replace a depth buffer. Lighting is baked; runtime materials are
unlit. Two 240×160 RGB565 field buffers consume 153,600 bytes. At 60 fields/s,
each individual LCD row refreshes at 30 Hz. The small on-screen number reports
field cadence, not complete progressive frames.
The hot cache has 128 slots / 1 MiB of RGB565 samples. Each slot covers a 64×64 region of a material/mip's 1024×1024 integer UV domain. Workers record sampled demand independently. Between completed renders, the application fills and replaces tiles under a soft 1 ms update budget. Partial tiles stay hidden. Hits match the current bilinear sampler exactly; misses use nearest. All cache allocations occur at startup, avoiding runtime allocation churn. Textures and palettes are immutable here; changing them would require invalidation.
On the tested S3, two full 90-second tours of the original camera route and palette pack reported 32.6–60.0 fields/s, with close window passes reaching 60 fields/s and 100% sampled cache hits. Free memory stayed at 34,243 bytes internal RAM and 1,858,852 bytes PSRAM after warmup, with no logged panic, watchdog or heap-corruption errors. The pool plus metadata allocates 1,084,416 bytes PSRAM and 8,216 bytes internal feedback storage. The largest recorded cache update was 3.239 ms: the budget is a target, not a hard deadline. Hit percentages describe short sampled windows, not a tour average.
For context, the same tour with live bilinear filtering reported 22.6–40.0 fields/s; nearest reported 39.2–60.0. Different moving-camera captures are not a matched microbenchmark. A separate fixed-view test found exact prefiltered bilinear reduced close-window update/render work from about 26 ms to 9–11 ms, with identical image hashes. These are historical measurements: the new palette pack has been visually approved on the S3. The revised lighting-focused camera route has also been flashed and readback-verified, with a clean 100-second playback capture. That validation is not a matched performance comparison. Validation details.
Turn CHAPEL_HOT_FILTER=OFF before enabling any fixed benchmark. Available
switches in main/CMakeLists.txt cover nearest/bilinear/three-point comparisons,
flash versus PSRAM, the original source-tile cache, and a fixed 2 MiB prefilter
proof. Without hot filtering, Chapel::defaultFilter selects three-point;
Chapel::setFilter can select nearest or ordinary bilinear. This tiled API and
the application-managed cache are experimental, not a general asset-streaming
system for arbitrary mutable textures.
Use CMake and a C++17 compiler (an MSVC developer prompt on Windows):
cmake -S esp32-streaming-chapel/tests -B build-native -DCMAKE_BUILD_TYPE=Release
cmake --build build-native --config Release
ctest --test-dir build-native -C Release --output-on-failureTests cover palette sampling, filtering, cache publication/eviction, perspective error, camera controls, framebuffer guards and serial/parallel rendering parity. The media instructions describe the native capture tool.
This is optional. The original project is fetched only for regeneration; it is not required by the firmware build. From this repository's root:
git clone https://github.com/lambertjamesd/n64brew2023.git upstream
git -C upstream checkout 8841ddf3e7d591af17287391f4b3b8728064c7bf
python -m pip install -r esp32-streaming-chapel/tools/requirements.txt
blender --background --factory-startup --disable-autoexec --python esp32-streaming-chapel/tools/export_scene.py
python esp32-streaming-chapel/tools/pack_assets.pyThe conversion was validated with Blender 4.4.3. The exporter disables embedded
script execution and reads the pinned scene. The asset manifest retains source
hashes; conversion changes are described in asset provenance.
Pillow 12.3.0 and NumPy 2.3.5 reproduce the S3-approved weighted palette pack
byte for byte in the validated environment. To use an existing upstream checkout,
pass --upstream /path/to/n64brew2023 to pack_assets.py. The manifest records
generator versions and source hashes; other library versions or platforms may
change numerical rounding or quantization results.
CubeCoders' example code is MIT. James D. Lambert's original chapel and textures retain his MIT notice, including when redistributed in converted form or pictured in this README. Please credit and visit the original N64 project.
Jet and LovyanGFX retain their licences in their pinned submodules. See THIRD_PARTY_NOTICES.md for the complete attribution map. The original project's audio and N64 binaries are not included.