Status: native drawing is implemented for authored GLSL ambient and per-light stages and for authored ARB assembly material stages, which are translated to GLSL. Unknown program names now load and compile their actual source pair. Stock-name overrides also use their actual source; an embedded stock implementation requires a reviewed source fingerprint or a canonical name with no source files. Vulkan remains experimental; the remaining work below covers broader language and state coverage.
src/renderer/materialprogram/GLSLCompiler.cpp accepts a vertex/fragment source
pair and explicit, case-sensitive material parameter and texture bindings. It
uses the pinned glslang 15.1.0 compiler in process, through a static library built
by the project's native Meson subproject. It does not invoke an installed SDK
executable, perform file access, or create GPU resources. Source inputs are
limited to one MiB each; empty and embedded-NUL inputs are rejected. Compilation
is serialized around the library's process lifetime and independent callers
can compile safely.
The original OpenGL GLSL pair is parsed and linked before translation. This is necessary because the compatibility layer initializes per-invocation globals from uniform storage; translating first could accidentally legalize an authored write to a uniform. The glslang preprocessor expands macros and conditionals. The translator operates on tokens, preserving diagnostic line records and function-local variable shadowing. Generated Vulkan shaders are linked together before emitting SPIR-V 1.3 for Vulkan 1.1. Failed compilation clears both output modules and returns a diagnostic, with the shader filename for source failures.
The initial language subset covers GLSL 1.10/1.20 scalar/vector float uniforms, 2D/cube samplers, named varyings, the engine's explicitly bound vertex attribute names, common legacy vertex/color/texture-coordinate inputs, model-view, projection, normal and texture matrices, fragment color and coordinates, and the supported 2D/cube explicit-LOD/gradient functions. Material names are not a shader-family whitelist. Uniform and interface arrays, authored matrix/int/bool uniforms, other sampler types, newer GLSL versions and other extensions remain unsupported and receive diagnostics. Unrecognized legacy built-ins must pass the Vulkan compiler; they are not silently replaced with dummy values.
The parameter capacity is the engine's 32 slots; the texture capacity is eight. Active scalar/vector parameters must match the material's upload width. The runtime draw interface is:
| Resource | Binding | Layout |
|---|---|---|
| Per-draw data | Set 0, binding 0 | 1,328-byte std140 block |
| Authored textures | Set 1, bindings 0–7 | Separate combined image samplers |
| Vertex inputs | Locations 0–5 | Position, color, normal, tangent, bitangent, UV |
The uniform block contains 32 parameter vectors, model-view and projection
matrices, eight texture matrices, stage color, four draw controls, two
CPU-composed model-view-projection matrices and a texture-orientation word
(1,328 bytes). Matrices are column-major. Images that Vulkan rendered keep
top-down rows; every authored 2D lookup on a declared sampler goes through a
generated helper that flips the coordinate (and the Y gradients) for the
slots the orientation word marks, so render targets sample the way OpenGL's
bottom-up rows do. A sampler passed through a function parameter keeps the
plain lookup. Authored
matrix operations retain GL clip coordinates; the generated vertex entry point
converts clip depth to Vulkan's range. Unmodified ftransform() or legacy MVP
positions use the same CPU-converted matrix as depth fill, preserving equal-depth
lighting coverage. Changed authored positions keep the ordinary clip conversion.
Window Y and dFdy select
the target's orientation through a pipeline specialization constant. In the authored GLSL
path, vertex color arrays are exposed directly, matching draw_common.cpp;
the wrapper does not invent stage tint or inverse modulation for those arrays.
The first native comparison covers texture coordinates, model-view/projection,
authored vertex offsets, fragment coordinates and derivatives in the ordinary
display path. Other target, color and geometry combinations remain unqualified.
OpenGL and Vulkan share the source-pair search order in
MaterialProgramSource.cpp. Vulkan passes the complete file lengths to the
compiler, preserving embedded-NUL rejection. Shader identity includes the
program name, parameter names/order/semantics and texture names/order. Numeric
values and image selection remain per draw. reloadGLSLprograms invalidates the
authored source lookup on both backends; reloadARBprograms also performs this
GLSL reload. Vulkan reuses unchanged valid or invalid source pairs before
checking its version limit. Changed
versions stay alive until device shutdown so recorded draws cannot reference a
destroyed module or a reused pipeline identity.
Native stock selection uses material_program_sources.json and the generated
vk_MaterialSourceSignatures.h. Both source files must match one reviewed pair;
only CRLF line endings are normalized. The table contains hashes, not retail
shader text. tools/build/vulkan_material_sources.py --check is registered in
the native Meson tests and script CI. It compares the generated table with the
reviewed manifest and never automatically approves changed shader source.
Different directories with the same basename are authored programs. Modified,
empty and incomplete pairs cannot silently select a stock implementation.
Some retail guide families have no GLSL source files; their canonical names
retain the existing built-in default only while every candidate source is
absent. Source selection is cached until an explicit shader reload or device
shutdown. A complete pair retains the shared filesystem search order.
OpenGL reload invalidates parsed material stages and the engine's material GLSL stages. Failed attempts are cached in the current GL context, so a broken file does not compile again on every draw. Explicit reload or a new context retries it. Neither backend needs a game restart to load a repaired supported shader.
Each frame slot owns a descriptor pool reset only after its fence completes. The larger uniform block uses the existing frame ring with the device's uniform alignment. All eight texture descriptors use the existing image residency and transition path. Shader modules, layouts, pools and CPU records are destroyed with the device, after the executor destroys its pipelines. GL handle fields are never used to store Vulkan handles.
Limits are explicit: 256 retained program versions, 32 MiB of retained compiler source/module data, 2,048 authored draws per frame slot, and the executor's shared 128-entry material pipeline cache. Allocation, type and unsupported binding failures diagnose and skip the affected stage. Exhaustion/recovery drills are still required; these bounds are not a performance claim.
Supported customLighting GLSL stages now execute once per contributing light
using their actual shader pair. Per-draw bindings include local light/view
origins, projected-light and falloff planes, bump/diffuse/specular texture rows,
diffuse/specular colors, vertex-color modulation, model/projection rows and the
identity color matrix. Light projection/falloff and other engine texture
semantics resolve from the current interaction. Stage conditions, colors and
texture transforms remain material-controlled. Draws accumulate with ONE/ONE
blending and inherit the receiver's geometry, depth, cull, bias and stencil state.
Arbitrary authored shaders do not have the engine's native shadow-map sampler contract. When mapped shadows are requested, only affected receivers run through a separate stencil-filtered interaction walk; other surfaces retain their mapped shadows. This matches OpenGL's retail stencil caster policy and does not add support for expanded map-only casters. The front end conservatively retains volumes for custom or unresolved interactions before submitting any receiver. The cost of that conservative retention still needs performance qualification. The two verified native lighting families keep their existing mapped path.
Explicit linear/nearest image filters now select the base mip level even when an image already has a mip chain. This matches OpenGL's sampler behavior across image reload. The lighting investigation also repaired OpenGL's material texture table: unused declared array slots now receive valid material images instead of retaining a comparison-enabled shadow texture from a previous pass.
src/renderer/materialprogram/ARBTranslator.cpp translates
ARB_vertex_program and ARB_fragment_program 1.0 material programs into the
compatibility GLSL above, so they reuse the same compiler, descriptor layout,
pipeline cache and draw path. The section is chosen exactly as
R_LoadARBProgram chooses it (the first !!ARBvp/!!ARBfp up to the first
END), so a program OpenGL truncates is rejected the same way. Every
instruction of both languages is supported, including _SAT, KIL, SWZ,
TXP/TXB, ARL relative addressing, PARAM arrays and ranges, ALIAS,
OUTPUT and ARB_position_invariant (which keeps the depth fill's exact
position). Matrix state binds the model-view, projection, MVP and texture
matrices with their inverse/transpose/invtrans rows. A stage with only
one program gets the fixed-function stage it replaces: identity texture
matrices with unit 0's coordinates, or the interpolated color.
The parameters are the ones draw_common.cpp gives ARB material stages:
vertexParm feeds both programs' locals unless fragmentParm is present;
locals the material does not set read zero. The vertex environment is env 0
(current-render scale), env 1 (global eye) and env 5-8 (local eye and model
rows); the fragment environment is env 0 and env 1 (window-to-texture scale).
Textures bind by unit from fragmentMap, with their own sampler state.
Anything a material stage has no defined value for is rejected with a
diagnostic rather than replaced: fixed-function lighting, fog, texture
generation and other legacy state, other environment indices, vertex.weight
and attributes the engine does not supply, 1D/3D/RECT targets, a unit
used with two targets, ARB_fog_* and NV options.
Programs without a reviewed native family are translated automatically, as
are pairs mixing families or naming an authored program on either side.
r_vkARBTranslation 0 restores the old skip; 2 also translates the stock
families, as a diagnostic comparison against their native implementations.
reportShaderPrograms lists each registered program as native, translated or
unsupported.
The native openq4-arb-translator test translates and compiles original
fixtures covering every instruction, the fixed-function counterparts and 28
rejection cases, and with --arb-dir the installed game's own .vfp files
(their material programs translate; the interaction programs are correctly
refused). tools/tests/renderer_vulkan_arb_programs.py draws eight original
programs in the PBR laboratory on both backends: every one of its 15 controls,
including image and shader reload and partial and full restart, matches
OpenGL's native ARB execution within one display level. The retail
heatHazeWithMask.vfp translated with r_vkARBTranslation 2 also draws
validation-clean on game/airdefense1.
The native openq4-material-compiler test covers valid paired programs,
parameters at the capacity boundary, 2D/cube textures, named/legacy attribute
aliases, normal transforms, macros, local variable shadowing, derivatives,
explicit LOD, deterministic recompilation and concurrent callers. Negative cases
cover syntax and link failures, uniform writes, invalid/duplicate bindings,
unsupported interfaces and source-size constraints. An independent spirv-val
check validates the emitted modules. These checks establish compilation and
module validity; they are not image, performance, or gameplay qualification.
The v50 Windows native-Meson run passed all 31 cases and the registered Meson
test. The v51 suite adds two invalid parameter-width cases and passes 33 cases.
Independent validation passes all 24 emitted modules and verifies the
uniform member offsets, matching vertex/fragment interfaces, boundary texture
slots and deterministic recompilation. The latest compiler evidence is retained
in .tmp/vulkan-gap-closure/authored-glsl/compiler-qualification.json; the
prior foundation evidence remains in the material-programs directory.
All 2,978 extracted upstream files and both overlay files were compared with
their pinned inputs. Other operating systems have not yet run this compiler
suite. The v51 runtime links the compiler into the Vulkan module; it requires
the matching staged engine and renderer modules. The prior PBR cutout fix is
retained, and its separate filtering discrepancy remains open.
The expanded v51 scene uses two independently compiled material programs and
engine screenshot captures in hidden, windowed, input-disabled gameplay. The
ten-control v51a run passes GL/Vulkan comparisons within one display level;
image reload, partial restart and full restart reproduce each backend's base
image exactly. Its unchanged-source shader control did not detect that Vulkan
had not registered the reload command; v52 adds actual live-edit coverage.
The reusable harness is
tools/tests/renderer_vulkan_material_programs.py; reports remain under
.tmp/vulkan-gap-closure/authored-glsl/. This is local image evidence, not
stock-map, performance, long-session or platform promotion evidence.
The capacity scene exposed a separate GL reference bug: ambient GLSL stages linked the named tangent/bitangent/normal attributes to lanes 9/10/11 without enabling and supplying those arrays. Those stages now bind the surface basis and legacy normal array around the draw. The capacity fixture uses all three named basis vectors to choose cube faces; a constant-color cube is a separate control. This preserves a meaningful reference instead of reproducing stale attribute values in Vulkan.
After that repair, all twenty v51d and final v51e controls pass. The 32-parameter/eight-texture
program, the changed last parameter and each independently changed texture slot
match GL pixel for pixel. The remaining controls differ by at most one 8-bit
display level, within the unchanged limit of two. Both backends show more than
29,000 changed pixels for each binding control and exact recovery images. The
tests cover named basis attributes, scalar/vec2/vec3/vec4 parameters and 2D/cube
samplers. These results do not establish untested language, blend or target
combinations. Failed intermediate reports remain retained with their diagnostics.
The final staged build also passes the render-target/startup, HDR and temporal
motion GPU regression groups with zero tracked warnings. Runtime, sources,
compiler outputs and reports are bound by
.tmp/vulkan-gap-closure/authored-glsl/checkpoint-v51-authored-glsl.json.
The v52 live-edit suite, tools/tests/renderer_vulkan_material_reload.py, adds
sixteen controls per backend. It exercises a stock basename under a different
directory, a canonical stock source replacement, changed fragment and vertex
source, syntax failure and repair, a fragment-only canonical source pair and
repair, both reload commands, and partial/full video restart. All sixteen
GL/Vulkan pairs differ by at most one display level; each backend's recovery
and deliberate failure images match their references exactly. The captures
also require the three unchanged SMAA sources to select verified native code.
The two deliberate failures each produce exactly one primary diagnostic before
repair, including a subsequent still-invalid capture. Expected compiler
messages are accepted only inside explicit bounds; unrelated warnings, API
errors and unknown commands still fail the run. Raw logs remain retained.
This suite found that the earlier Vulkan reload functions had no registered
commands: registration was inside OpenGL initialization. Both commands now
register in shared renderer initialization. The first failed Vulkan run is
retained, and the common laboratory now rejects unknown commands. The initial
GL diagnostic parser also needed to strip engine color codes, and the Vulkan
parser needed the specific follow-on compiler messages for the deliberately
undefined identifier. These parser failures remain failed reports. Accepted
source-edit evidence is gl-v52b, vk-v52c and compare-v52b under
.tmp/vulkan-gap-closure/material-reload/.
The final staged runtime also passes 35 OpenGL HDR/bloom controls, including
exact full-frame restoration after both GLSL reload commands. The Vulkan
render-target/startup, HDR and temporal-motion GPU groups pass without tracked
warnings. Both registered native tests pass; the 33-case compiler output is
byte-identical to all 24 independently validated v51 modules. The implementation
and retained evidence are bound by
.tmp/vulkan-gap-closure/material-reload/checkpoint-v52-material-reload.json.
The v53 fixture adds 26 authored-lighting controls with 27 semantic bindings,
point/projected lights, changed projection/falloff textures, material transforms,
vertex colors, conditions, multiple stages/lights, rotation and shadow/recovery
controls. Both v53e backend runs pass independently with exact image/shader/video
recovery and exact mapped-to-stencil equivalence on the custom receiver. Every
receiver comparison stays within one display level, including exact shadowed
receiver output. Projected-shadow full frames differ by at most one level. The strict
full-frame comparison still fails for three point-stencil background pixels
(maximum 30 levels) and nineteen point-map pixels (maximum five). The receiver
analysis only locates these failures; the original full-frame gate remains
failed. Reports and engine captures are retained under
.tmp/vulkan-gap-closure/authored-lighting/.
Development runs exposed fixture projection/removal errors, canonical transform
rounding holes, an overly restrictive stencil-admission check, the OpenGL
sampler-table conflict and a one-pixel Vulkan image-reload change. Those reports
remain failed evidence. The v53e sampler correction closes the last exact-recovery
failure without changing the threshold. Compiler qualification passes all 33
native cases and independently validates 24 newly generated modules with the
expanded uniform block; see authored-lighting/compiler-qualification.json.
The final v53e runtime also passes twenty ordinary authored-material comparisons
within one display level, with exact recovery, all 35 OpenGL HDR/bloom controls
and the three Vulkan render-target/startup, HDR and temporal-motion GPU groups.
These runs report no tracked API/engine diagnostics. Source/runtime identity,
the passing scoped checks and the failed full-frame comparison are retained in
.tmp/vulkan-gap-closure/authored-lighting/checkpoint-v53-authored-lighting.json.
The v56 point-shadow correction retains all 26 independent lighting controls and twenty ordinary material comparisons. Its complete mapped-shadow image now stays within two display levels of the unchanged GL reference. Three point-stencil background pixels still fail, so the full lighting comparison remains failed overall. This follow-up does not extend authored-shader admission or weaken the image gate.
Remaining work includes:
- Extend source-selection qualification to additional retail/mod source pairs, filesystem overlays, and long repeated-edit sessions.
- Qualify additional custom-lighting combinations, including mixed stages, translucent receivers, alpha coverage, animated geometry and resource failure. The generic evaluator now handles the per-light semantics above in addition to numeric registers, view origin, current render viewport/scale and supported post-process size/color-space bindings. Full-scene shadow parity remains open.
- Qualify offscreen/HDR, texture generation, alpha and blend state, vertex colors, decals and animated geometry for authored programs, including flipped render-texture sampling in a real capture scene.
- Extend the GLSL language subset and qualify descriptor/pipeline/uniform/cache exhaustion, repeated source edits and device-failure recovery.
- Qualify translated ARB programs against real mod content and the
_currentRendercontract on a post-process scene (the laboratory renders through the fast no-post path, so its fixtures do not sample it).
The broader HDR/PBR/shadow, filtering, performance, CI, hardware and release requirements in the gap-closure plan remain in force.
What draws. A stock GLSL or ARB pair whose sources match a recorded
signature draws through its native Vulkan family. Every other program stage is
authored: GLSL compiles in process, ARB is translated first, and a valid
version draws through the generic executor. r_vkARBTranslation 2 also
translates stock pairs as a diagnostic; when that translation fails, the native
family still draws and the log says so once per material.
What is skipped, and how you see it. The unit of fallback is the stage.
An authored stage that cannot draw is skipped, and the rest of the material and
frame draw normally. Nothing substitutes another program or a flat colour. Each
skip is reported once per program and reason until the next reload: compile or
translation failure, missing or mismatched texture, unavailable fragmentMap,
pipeline, uniform-ring or descriptor exhaustion, or no open rendering scope.
These lines replace the earlier single warning per program, which hid every
later reason. An authored ARB material that cannot draw at all is named once
per material, not once per session. The set of reported pairs is bounded at
512; later pairs are counted as suppressed. reportShaderPrograms now lists
every version as compiled or invalid, current or stale, with its draws and skips
since the last reload, the last skip reason and the compiler diagnostic.
GL parity for shader parameters. OpenGL's RB_BindGLSLShaderParm declines a
binding it has no value for, such as a per-light vector on an ambient stage or a
sample array, and still draws the stage: the parameter takes its registers or
keeps its default. Vulkan now does the same instead of skipping the stage.
Per-light bindings outside an interaction stay quiet, as on GL; any other
binding with no Vulkan value is reported once. Authored pipelines also honour
the stage's alpha-to-coverage bit, as GL's GL_State and the native families do.
Bounded resources.
| Resource | Limit | At the limit |
|---|---|---|
| Registered ARB program names | 1,024 (GL: 200, with 48 built-ins) | Warns once per name; the declaring stage is skipped. It no longer ends the session. |
| Authored program versions | 256 | A new version evicts the least recently drawn version that no current key names and that no frame in flight can still use. Its shader modules and pipelines are released. If every version is current or in flight, the stage is skipped and retried once the frames have retired; its sources are not re-read on every draw. |
| Compiler storage | 32 MiB | The version is invalid ("storage exhausted"); evictions return their bytes. |
| Authored draws per frame | 8,192 (four chained pools of 2,048) | The draw is skipped and reported. |
| Program pipelines (native and authored) | 512 (was 128) | Reported once; later program stages are skipped. |
| Per-frame vertex, index and uniform rings | 4 MiB uniform | Reported at the first overflowing frame and at each power of two, naming the ring. The old message called every ring "geometry". |
Reload. reloadGLSLprograms and reloadARBprograms clear the lookup state,
the skip counts and the reported pairs. Each version is re-validated on its next
use, and an unchanged source reuses its version, including an invalid one,
which reports its diagnostic again once. Repeated edits no longer exhaust the
table: stale versions are evicted as above.
The three authored-program laboratories ran against a clean build of
fef4b5875 (0.13.2-dev+gfef4b587, dirty 0): a detached worktree built and
staged as one debugoptimized runtime, then a lean laboratory copy with the
laboratory map compiled by its client. Each suite captured an OpenGL reference
and a Vulkan candidate with validation enabled, then ran its own comparison.
| Suite | Controls | GL | Vulkan | GL/Vulkan comparison |
|---|---|---|---|---|
renderer_vulkan_material_programs.py (authored GLSL, 32-parameter/8-texture capacity program, image/shader reload, partial/full restart) |
20 | pass | pass | pass: maximum one display level, 11 exact |
renderer_vulkan_material_reload.py (source selection, live vertex/fragment edits, syntax failure and repair, partial pair, both reload commands, restarts) |
16 | pass | pass | pass: maximum one display level, 5 exact |
renderer_vulkan_arb_programs.py (translated ARB programs, reload and restarts) |
15 | pass | pass | pass: maximum one display level, 5 exact |
The comparison limit is two display levels. Within each backend, the recovery and deliberate-failure images matched their references exactly; the reload suite's two deliberate failures each produced exactly one primary diagnostic. The Vulkan logs contain no validation messages and none of the new skip or approximation reports, so the bounded version table's reuse, repair and new-version paths all drew correctly.
The first OpenGL capture of the GLSL suite was killed by the harness's 600 s
default after 19 of 20 captures, during the final full restart, while another
session's native tests loaded the machine. It is retained as
gl-timeout-600s; the rerun with a longer limit took 123 s. One reload capture
was interrupted when the runner paused for another session and was repeated.
Evidence: .tmp/vkrm/labs/ in the main checkout.
Not covered: eviction at 256 versions and the other capacity limits, which
these suites do not reach; the authored-lighting laboratory
(renderer_vulkan_material_lighting.py), whose strict full-frame gate still
fails on three point-stencil background pixels; and a mod-content corpus.
glslang 15.1.0 is pinned by
archive SHA-256 in subprojects/glslang.wrap. Its BSD, MIT, Apache-2.0, GPLv3 with
Bison exception and permissive preprocessor notices were reviewed for this
GPLv3 repository before incorporation. The native Meson overlay selects the
upstream compiler/SPIR-V/resource-limit sources, without changing their source
or notices. Optional HLSL, SPIRV-Tools, remapper and upstream test dependencies
are excluded. The complete upstream license is retained in
LICENSES/glslang.txt and included in the staging manifest under
licenses/; the README credits the original project.