Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,010 changes: 1,010 additions & 0 deletions assets/shadow_residency/ShadowResidencyCasterMotionTest.gltf

Large diffs are not rendered by default.

1,010 changes: 1,010 additions & 0 deletions assets/shadow_residency/ShadowResidencyLightMotionTest.gltf

Large diffs are not rendered by default.

81 changes: 70 additions & 11 deletions assets/shadow_residency/generate.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,11 +98,24 @@ def scaled(axis, distance):
return tuple(component * distance for component in axis)


def build():
def build(motion=None):
"""The room, the light and the six casters.

`motion` selects which of the three scenes this is: `None` for the static gate,
`"caster"` for one caster moving inside a single cube face, `"light"` for the light
itself moving. They are separate FILES rather than phases of one animation, and
deliberately: the diagnostic evidence is a per-family count (`recorded=1 reused=5`
against `recorded=6 reused=0`), and a scene that changed regime part-way would make
every aggregate a mixture of two answers with no way to say which frame is which.
"""
s = Scene(GENERATOR)

# Node indices are captured as they are created — `Scene` hands one back from every
# builder — because the animation channels below address nodes by index.
index = {}

# The light first, so it is light 0 and the scene's only one.
s.add_node(
index["PointLight"] = s.add_node(
"PointLight",
light=s.light(
"Point",
Expand All @@ -121,18 +134,49 @@ def build():
colour = FLOOR_COLOUR if name == "NegY" else WALL_COLOUR
s.box(f"Wall{name}", wall_half_extent(axis), wall_centre, colour)
# One occluder per face, on the axis between the light and that wall.
s.box(
index[f"Caster{name}"] = s.box(
f"Caster{name}",
(CASTER_HALF, CASTER_HALF, CASTER_HALF),
scaled(axis, CASTER_DISTANCE),
CASTER_COLOUR,
)

s.camera(CAMERA_EYE, CAMERA_TARGET)

if motion == "caster":
# CONTINUOUS, and confined to ONE face. Radial motion along the +X axis keeps the
# caster inside the +X face's 90-degree frustum for the whole loop while changing
# what that face stores every frame — so the steady state is `recorded=1 reused=5`,
# which is the per-face granularity this item is about, stated as a number.
#
# Continuous rather than a single hop: the diagnostic sample is periodic, and a
# one-off transition can fall between two samples and read as a scene that never
# changed.
near = CASTER_DISTANCE - 0.8
far = CASTER_DISTANCE + 0.8
s.animation(
"CasterInOneFace",
[
(index["CasterPosX"], "translation", [0.0, 2.0, 4.0],
[(near, 0.0, 0.0), (far, 0.0, 0.0), (near, 0.0, 0.0)], "LINEAR"),
],
)
elif motion == "light":
# The light itself moves, so every face's stored depth changes: its position is an
# input to the radial ratio each of the six faces writes, not merely to the matrices.
# Steady state is therefore `recorded=6 reused=0` — the honest ceiling for what
# punctual reuse can save when the light is the thing in motion.
s.animation(
"LightSweep",
[
(index["PointLight"], "translation", [0.0, 2.5, 5.0],
[(-1.0, 0.0, 0.0), (1.0, 0.0, 0.0), (-1.0, 0.0, 0.0)], "LINEAR"),
],
)
return s


def validate(doc):
def validate(doc, motion=None):
"""Structural checks, because this scene's whole value is what it does NOT contain.

A later edit that adds a sun, or animates a caster to make a screenshot livelier, would
Expand All @@ -148,7 +192,18 @@ def validate(doc):

# Temporal or deforming content would change the content descriptor every frame, so a
# reused view could never happen and the gate would fail for the wrong reason.
assert not doc.get("animations"), "the residency gate scene must not animate"
if motion is None:
assert not doc.get("animations"), "the static gate scene must not animate"
else:
# Exactly ONE animated node, and the right one. A second channel would mix two
# regimes into one family count and make `recorded=N` unreadable.
animations = doc.get("animations", [])
assert len(animations) == 1, f"expected one animation, found {len(animations)}"
channels = animations[0]["channels"]
assert len(channels) == 1, f"expected one animated node, found {len(channels)}"
animated = doc["nodes"][channels[0]["target"]["node"]]["name"]
expected = "CasterPosX" if motion == "caster" else "PointLight"
assert animated == expected, f"{motion} scene animates '{animated}', not '{expected}'"
assert not doc.get("skins"), "the residency gate scene must not contain skinned casters"
for mesh in doc["meshes"]:
for primitive in mesh["primitives"]:
Expand Down Expand Up @@ -189,12 +244,16 @@ def validate(doc):


def main():
scene = build()
doc = scene.to_gltf()
validate(doc)
out = Path(__file__).resolve().parent / "ShadowResidencyTest.gltf"
write_gltf(out, doc)
print(f"wrote {out}")
here = Path(__file__).resolve().parent
for motion, name in ((None, "ShadowResidencyTest.gltf"),
("caster", "ShadowResidencyCasterMotionTest.gltf"),
("light", "ShadowResidencyLightMotionTest.gltf")):
scene = build(motion)
doc = scene.to_gltf()
validate(doc, motion)
out = here / name
write_gltf(out, doc)
print(f"wrote {out}")


if __name__ == "__main__":
Expand Down
43 changes: 43 additions & 0 deletions docs/acceptance-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -591,6 +591,49 @@ Look at the image as well as the hash. The two large hard-edged rectangles on th
cast shadows; if a run ever produces an image with a shadow in the wrong place rather than a
different hash, that is a stale map and `--no-shadow-reuse` will confirm it in one run.

### 4. Per-face granularity — the punctual payoff (arc 2 #15)

The static gate above proves reuse happens; these two scenes prove what it is WORTH when something
is actually moving, which is the case a shipping scene is in. They are separate files rather than
phases of one animation on purpose: the evidence is a per-family count, and a scene that changed
regime part-way would make every aggregate a mixture of two answers.

```bash
# One rigid caster moving INSIDE a single cube face; the light and the other five casters are static.
FE_LOG=render:debug ./fireEngineApp --no-taa \
shadow_residency/ShadowResidencyCasterMotionTest.gltf nightbox.hdr
```

Steady state must be **`point sampleable recorded=1 reused=5 passes=1`**. One face's content changed,
so one face re-rendered; the other five are still resident and still sampleable. Then the same scene
with the cache off:

```bash
FE_LOG=render:debug ./fireEngineApp --no-taa --no-shadow-reuse \
shadow_residency/ShadowResidencyCasterMotionTest.gltf nightbox.hdr
```

`recorded=6 reused=0 passes=6`. Measured here (macOS/arm64, MoltenVK, 15 warm samples each):
**0.007 ms** median for the point family with reuse (0.004–0.022) against **0.141 ms** forced
(0.124–0.192) — the same scene, the same image, a factor of about twenty, and a direct measurement
of per-face granularity rather than of reuse in general.

```bash
# The light itself moving — the ceiling on what punctual reuse can save.
FE_LOG=render:debug ./fireEngineApp --no-taa \
shadow_residency/ShadowResidencyLightMotionTest.gltf nightbox.hdr
```

Expect **`recorded=6 reused=0`** (measured: 0.172 ms median, 9 samples) even with reuse enabled,
and that is correct rather than a failure:
a point light's position is an input to the radial depth EVERY face stores, so a light that moves
invalidates its whole cube. A run that showed anything less would mean a face had kept depth measured
against a position the light has left.

**No reference screenshot for either motion scene.** An animated frame has no reproducible timestamp,
so a capture proves nothing; the State column, the family counters and the timing comparison are the
evidence.

### 4. Vulkan validation stays clean

Add `--require-validation` to any of the above. A reused view is skipped **entirely** — no barrier,
Expand Down
2 changes: 1 addition & 1 deletion docs/architecturalreview.md
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ exists; the tiered review inherited the filename.)*
| 12 | ✅ Skip frustum extraction for inactive shadow slots *(same branch; coarse-cull pushes only active cascade/spot/point slots — also tightens the cull)* | C | XS | §5.3 |
| 13 | ✅ Comment: chart-set immutability rationale *(same branch)* | C | XS | §4.2 |
| 14 | ✅ Comment: hinge axis rows prepare-frozen by design *(same branch)* | C | XS | §3.4 |
| 15 | Punctual-shadow change detection (spot/point re-render every face every frame; point is 6×1024² each) — same epoch/dirty-bit mechanism as item 4 | B/C | M | §2.3 |
| 15 | Punctual-shadow change detection *(branch `shadow-punctual-change-detection`)* — verified, closed with NO engine change. Arc 2 #4's residency mechanism is family-agnostic and already covered spot and point; this item's value was the evidence. Audit: every shadow sampling input is re-uploaded from the view set each frame, every raster input is in the content descriptor, nothing is outside both. Pinned headlessly: range-only changes re-record a whole cube, a moving light re-records all six faces (its position is an input to every face's radial depth), an inherited slot re-records rather than reusing a predecessor's map, and bias metrics are sampling rather than content. Measured: a caster moving inside one face holds `recorded=1 reused=5` at 0.007 ms against 0.141 ms forced. Slot churn is parked behind runtime scene mutation. | B/C | M | §2.3 |
| 16 | `hash_combine`-style mix for the mesh-triangle warm-start key (XOR admits collisions across (pair, triangle); consequence is a wrong warm-start seed, self-correcting) | C | XS | §3.3 |
| 17 | Retire the TAA resolve→blit full-res 16F copy by treating `history[cur]` as the scene target — **measure on MoltenVK first**; costs per-frame or double-buffered descriptors for three consumers | C | M | §2.5 |
| 18 | `vdpmDrawCounts_` / `findSelfShadowViewProj` linear scans are O(fronts²) — a watch-item, not a defect at today's front counts | C | XS | §1.6 |
Expand Down
41 changes: 15 additions & 26 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,32 +81,21 @@ The eight small/XS items landed on `review-shadow-taa-fixes` + `review-xs-cleanu
the five the review prioritised, then five a later coverage audit found had no action item. In the
review's priority order:

- **#15 [B/C, M] Punctual-shadow change detection** (§2.3) — **next, and now mostly verification.**
Arc 2 #4 landed the whole mechanism on `shadow-residency-reuse` and it is family-agnostic: a
static point light's six faces already reuse today, which is what the gate scene measures. What
this item still owes is the evidence and the follow-through — a scene with a light that MOVES
(proving the faces re-record on the frame the light's position or range changes, since both are in
the content descriptor), a spot equivalent, and a decision about per-face granularity: the cube is
compared per face, but slot assignment is per-light, so a light entering or leaving reshuffles
slots and invalidates its neighbours' residency by identity. Worth measuring before assuming it
matters. (Per-face frustum filtering already exists and is correct.)
- **#5 [B/L] Compute pre-skinning pass** (§1.3) — skinning/morphing re-runs in every pass's vertex
shader (~11× per skinned vertex per frame). `SoftBodySystem` already proves the compute pattern
in-engine. The one genuinely architectural piece here; it also retires SH-04's deformable
full-detail fallback by exposing pre-deformed vertices + exact deformed bounds + a deformation
revision.
- **#7 [B/S] Physics per-step scratch persistence** (§3.1) — remove the per-step heap allocation in
the solver hot path. Golden-neutral if done as pure allocation reuse.
- **#10 [B/S] Front-to-back sort of the opaque bucket** (§1.1) — improves depth-prepass rejection.
- **#6 [C/S] Batch image barriers into single `DependencyInfo`s** (§1.2) — compounds on MoltenVK
(§5.2); coordinate with SH-* so barrier grouping doesn't change per-view LOD decisions.

**Added by a coverage audit** (2026-07-26). The review's §6 table was a *prioritised* list, not an
exhaustive one: five actionable findings in its body had no row. They are now rows 15–19 there and
items here. All five are genuinely lower-value than the above — three are conditional or watch-items
in the review's own words — and are recorded so the arc is scoped honestly, not because each is
worth doing:

- **#15 ✅ Punctual-shadow change detection** (§2.3) — verified on `shadow-punctual-change-detection`
and closed with **no engine change**, which was the honest outcome: arc 2 #4's mechanism is
family-agnostic and already covered the punctual families. What the branch added is the evidence —
an audit showing every shadow sampling input is re-uploaded from the view set each frame while
every raster input is in the content descriptor (so nothing sits outside both), eight headless
cases pinning the punctual specifics (range-only change, moving light, slot inheritance for both
spot and cube, metrics-are-not-content on both), and two motion scenes measuring the payoff:
`recorded=1 reused=5` at 0.007 ms against 0.141 ms forced, for an identical image. Detail in
[`shadowplans.md`](shadowplans.md) § Interaction; runbook in
[`acceptance-testing.md`](acceptance-testing.md).

**Parked, not done:** slot churn. A light leaving compacts punctual slots and every inherited slot
re-records a whole cube. Correct today (identity is compared) and unreachable in production (no
runtime light removal or enable/disable path, stable gather order), so a stable-assignment scheme
would guard against nothing. **Trigger: runtime scene mutation, or a light enable/disable toggle.**
- **#16 [C, XS] `hash_combine`-style mix for the mesh-triangle warm-start key** (§3.3) —
`in.key ^= subKey * 0x9E3779B97F4A7C15ULL` (`physics_world.cpp`) is a decent mix, but XOR over the
pair key admits collisions across (pair, triangle) combinations. The consequence is only a wrong
Expand Down
32 changes: 32 additions & 0 deletions docs/shadowplans.md
Original file line number Diff line number Diff line change
Expand Up @@ -1034,6 +1034,38 @@ contracts from this work:
`--no-shadow-reuse` (overlay: "Reuse unchanged shadow views"); the runbook is
[`acceptance-testing.md`](acceptance-testing.md) § Shadow-residency gate scene.

**Punctual change detection (arc 2 #15) is the same mechanism, verified rather than built.** The
audit that item owed is done, and its result is a boundary worth stating once: every shadow
SAMPLING input — `spotViewProj`, `cascadeViewProj`, `selfShadowViewProj` and all four
`*BiasMetrics` arrays — is copied wholesale from the completed view SET every frame,
unconditionally, never from the plan and never gated on whether a family recorded. Every RASTER
input is in the content descriptor. Nothing sits outside both, which is why a reused map cannot go
stale through a sampling parameter. (Shadow draws are also VDPM-free by construction — `object.cpp`
clears the indirect handle and the GPU front — so a per-frame GPU-emitted index buffer, whose
handle repeats while its contents change, can never enter the descriptor.)

The punctual specifics are pinned headlessly: a RANGE-only change re-records all six faces (every
matrix identical, every texel different — the case a transform-only descriptor would miss); a
moving light re-records all six, because its position is an input to the radial depth each face
stores; a cube inherited by another light re-records all six rather than lighting one light with
its predecessor's shadows; and metrics-only changes reuse, on both the spot and point paths.

**Per-face granularity is where the punctual saving actually lives, and it is caster-driven.**
A cube is compared face by face, but a light's own movement invalidates every face by the content
law, so only a CASTER can change one face and not another. Measured on
`ShadowResidencyCasterMotionTest` (one rigid caster moving inside a single face, 15 warm samples):
the point family holds `recorded=1 reused=5` at a median **0.007 ms**, against **0.141 ms** forced
on the same scene — about twenty times less for an identical image. The light-motion scene is the
ceiling: `recorded=6 reused=0`, 0.172 ms, which is what punctual reuse cannot save.

**Slot churn is correct and deliberately un-optimised.** Punctual slots are assigned per frame in
gather order, so a light leaving compacts the ones after it and every inherited slot re-records a
whole cube it could in principle have kept. That is SAFE — identity is part of the comparison, and
the tests above pin it — and its cost is currently unreachable: the scene graph has no runtime
light removal or enable/disable path, and `gatherLights()` walks a stable node order, so churn
frequency in production is zero. Revisit only when runtime scene mutation or a light enable toggle
arrives; until then a stable-slot-assignment scheme would be machinery guarding against nothing.

**What this does NOT buy: CPU preparation.** A reused view is filtered, resolved and observed in
full — the comparison cannot be made without the work that produces its operand. The saving is
GPU raster only, which is also why the honest headline case is a static PUNCTUAL light rather than
Expand Down
Loading
Loading