Feature/windowed vulkan device selection

AGENTS.md

@@ -7,13 +7,39 @@ AGAVE (Advanced GPU Accelerated Volume Explorer) is a C++17/Qt6 desktop applicat
 | Module            | Role                                                                                                                                              |
 | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `agave_app/`      | Qt6 GUI layer — widgets, dialogs, dock panels, OpenGL viewport (`GLView3D`)                                                                       |
-| `renderlib/`      | Core rendering engine — image I/O (`io/`), GPU pipeline (`graphics/`), camera, scene, gesture handling (`gesture/`), JSON serialization (`json/`) |
+| `renderlib/`      | Core rendering engine — image I/O (`io/`), graphics abstraction (`gfxapi/`) with OpenGL and Vulkan backends (`gfxOpenGL/`, `gfxVulkan/`), camera, scene, gesture handling (`gesture/`), JSON serialization (`json/`) |
 | `agave_pyclient/` | Python WebSocket client for remote control of AGAVE in server mode                                                                                |
 | `test/`           | C++ unit tests (Catch2)                                                                                                                           |
 | `webclient/`      | JavaScript client                                                                                                                                 |
 
 `agave_app` depends on `renderlib` for all rendering and data operations. Keep GUI concerns out of `renderlib`. `renderlib` should have no Qt dependencies and be testable in isolation. The Python client and web client communicate with the C++ engine via a binary command protocol defined in `renderlib/command.h` and implemented in `renderlib/command.cpp`. Commands must be added in all three locations to stay in sync (see "Adding a New Command" below).
 
+### Graphics backends (`gfxapi`)
+
+`renderlib/gfxapi/` is a backend-agnostic graphics abstraction so the rest of the
+renderer never talks to OpenGL or Vulkan directly. Its central interface is
+`gfxApi::Backend`, which owns the GPU device and any backend-global state and
+creates the concrete renderers, framebuffers, and contexts. `IGraphicsDevice`
+exposes the device-level primitives (shaders and shader programs today; the
+surface is deliberately minimal and grows as more primitives move behind it), and
+sibling interfaces cover the render window (`IRenderWindow`), gesture/manipulator
+drawing (`IGestureRenderer`), `Framebuffer`, `IGLContext`, and `IWindowSurface`.
+
+Concrete implementations live in `gfxOpenGL/` and `gfxVulkan/` (Vulkan is gated
+behind `AGAVE_HAS_VULKAN`; `WebGPU` is enumerated in `BackendKind` but not yet
+implemented). `renderlib::initialize` creates exactly **one** `Backend` for the
+process lifetime, chosen from `InitParams::backendKind`. The `createGraphicsBackend`
+function in `renderlib.cpp` is the single place that maps a `BackendKind` onto a
+concrete backend — the abstract `gfxapi/` layer must not depend on any concrete
+backend, and renderer code should reach GPU functionality through `Backend::device()`
+and the `gfxapi` interfaces rather than backend-specific APIs.
+
+Backend bring-up is two-phase: construction creates the instance/context, then a
+device is selected. Windowed Vulkan defers device selection to
+`Backend::initDeviceForWindow()` once a native surface exists; headless uses EGL
+(OpenGL) or `initDeviceHeadless()` (Vulkan). A null surface is only valid in
+headless mode.
+
 ## Build and Test
 
 Prerequisites and platform-specific setup are in [INSTALL.md](INSTALL.md). Dependencies are fetched via CMake FetchContent (GLM, Catch2) and require Qt 6.9.3 installed on the system.
@@ -99,7 +125,7 @@ clang-tidy -p build --fix renderlib/RenderSettings.cpp
 
 ## Adding a New Command
 
-Commands are the binary protocol connecting the C++ engine, Python client, and web client. Every command must be added to all four locations to stay in sync.
+Commands are the binary protocol connecting the C++ engine, Python client, and web client. Every command must be added with the following steps to stay in sync.
 
 ### 1. `renderlib/command.h` — declare data struct + command class
 
@@ -195,9 +221,53 @@ set_foo(x: number, mode: number) {
 }
 ```
 
+Steps 9–11 also make the command available in `agave_pyvk`, the in-process,
+headless Vulkan package. It does **not** serialize command buffers: `AgaveRenderer`
+passes typed arguments straight into `renderlib` through nanobind, and the C++ side
+dispatches on the integer command ID via `PythonRenderer`. Because that dispatch is
+data-driven, `agave_pyvk/src/bindings.cpp` is generic and needs **no** per-command
+changes.
+
+### 9. `renderlib/PythonRenderer.cpp` — register in both switches
+
+```cpp
+// In commandArgumentTypes(): maps ID -> argument type list.
+COMMAND_ARGUMENT_TYPES(52, SetFooCommand);
+
+// In execute(): constructs the command from positional args and runs it.
+// The count and the arg<T>(args, N) order/types must match the command's CMD_ARGS.
+EXECUTE_COMMAND(52, 2, SetFooCommand, SetFooCommandD{ arg<float>(args, 0), arg<int32_t>(args, 1) });
+```
+
+### 10. `agave_pyvk/agave_pyvk/commandbuffer.py` — add to `COMMANDS`
+
+This module keeps only the ID (no argument-type list); it exists for API/source
+parity with `agave_pyclient`.
+
+```python
+"SET_FOO": 52,
+```
+
+### 11. `agave_pyvk/agave_pyvk/agave.py` — add method to `AgaveRenderer`
+
+Mirror the `agave_pyclient` method, but dispatch through `self._execute`, which
+calls the native renderer synchronously and returns its result.
+
+```python
+def set_foo(self, x: float, mode: int):
+    return self._execute("SET_FOO", x, mode)
+```
+
 **Key rules:**
 
-- The integer ID must be unique and match across all four locations
+- The integer ID must be unique and match across all locations, including
+  `renderlib/command.h` and both `PythonRenderer` switches
 - Argument types are `F32`, `I32`, `S` (string), `F32A` (float array), `I32A` (int array)
 - Python method name uses snake_case; `COMMANDS` dict key is UPPERCASE
-- `parse()`/`write()` field order must match the `CMD_ARGS` type list exactly
+- `parse()`/`write()` field order must match the `CMD_ARGS` type list exactly, as
+  must the `EXECUTE_COMMAND` arg count and `arg<T>()` order/types
+- `agave_pyvk/src/bindings.cpp` is generic — do not add per-command code there
+- The public method must exist in **both** `agave_pyclient/agave.py` and
+  `agave_pyvk/agave.py`. `test_public_api_matches_pyclient` in
+  `agave_pyvk/tests/test_api.py` enforces that every `agave_pyclient` method is
+  also present on `agave_pyvk` (which only adds `load_array`)

agave_app/CacheSettingsWidget.cpp

@@ -55,6 +55,10 @@ CacheSettingsWidget::CacheSettingsWidget(QWidget* parent, AgaveSettingsData* set
   layout->addRow(QString(), m_applyButton);
   layout->addRow(QString(), m_clearDiskButton);
   setLayout(layout);
+
+  // Populate the fields from the already-loaded settings; otherwise the spin
+  // boxes show their default 0 until something else triggers a refresh.
+  updateUiFromSettings();
 }
 
 void

agave_app/QtVulkanSurface.h

@@ -2,7 +2,7 @@
 
 #if AGAVE_HAS_VULKAN
 
-#include "renderlib/gfxVulkan/Swapchain.h"
+#include "renderlib/gfxapi/WindowSurface.h"
 
 #include <cstdint>
 
@@ -17,7 +17,7 @@ class QWidget;
 // widget own the native surface keeps it laid out by Qt exactly like any other
 // widget, avoiding the geometry offsets that come with embedding a separate
 // QWindow via QWidget::createWindowContainer.
-class QtVulkanSurface : public gfxvulkan::ISwapchainSurface
+class QtVulkanSurface : public gfxApi::IWindowSurface
 {
 public:
   explicit QtVulkanSurface(QWidget* widget);

agave_app/VulkanView3D.cpp

@@ -7,6 +7,7 @@
 #include "ViewerState.h"
 
 #include "renderlib/AppScene.h"
+#include "renderlib/Logging.h"
 #include "renderlib/MoveTool.h"
 #include "renderlib/RenderSettings.h"
 #include "renderlib/RotateTool.h"
@@ -20,6 +21,7 @@
 #include <cstddef>
 #include <cstdint>
 #include <memory>
+#include <stdexcept>
 
 #include <QApplication>
 #include <QEvent>
@@ -72,7 +74,6 @@ VulkanView3D::VulkanView3D(QCamera* cam, QRenderSettings* qrs, RenderSettings* r
   : QWidget(parent)
   , m_qcamera(cam)
   , m_qrendersettings(qrs)
-  , m_viewerWindow(std::make_unique<ViewerWindow>(rs))
 {
   // Render directly into this widget's own native surface. Qt lays out a native
   // widget exactly like any other, so the Vulkan content aligns with its place
@@ -89,7 +90,29 @@ VulkanView3D::VulkanView3D(QCamera* cam, QRenderSettings* qrs, RenderSettings* r
   setMinimumSize(256, 256);
   setSizePolicy(QSizePolicy::Expanding, QSizePolicy::Expanding);
 
+  // Force Qt to realize the native window now. The backend has to build a real
+  // VkSurfaceKHR from it before it can tell which physical device and queue
+  // family can present here.
+  (void)winId();
   m_surface = std::make_unique<QtVulkanSurface>(this);
+
+  // Surface-aware device selection has to happen before ViewerWindow, which
+  // immediately creates the renderers and gesture renderer -- and those need a
+  // selected physical device, logical device, queues, and command pool.
+  gfxApi::Backend* backend = renderlib::graphicsBackend();
+  if (!backend) {
+    const auto msg = "Cannot create a Vulkan view without an initialized graphics backend";
+    LOG_ERROR << msg;
+    throw std::runtime_error(msg);
+  }
+
+  if (!backend->initDeviceForWindow(m_surface.get())) {
+    const auto msg = "Failed to initialize a graphics device that can present to the 3D view window";
+    LOG_ERROR << msg;
+    throw std::runtime_error(msg);
+  }
+
+  m_viewerWindow = std::make_unique<ViewerWindow>(rs);
   m_swapchain = std::make_unique<gfxvulkan::Swapchain>(m_surface.get());
 
   m_viewerWindow->gesture.input.setDoubleClickTime(static_cast<double>(QApplication::doubleClickInterval()) / 1000.0);
@@ -126,16 +149,6 @@ VulkanView3D::sizeHint() const
   return QSize(800, 600);
 }
 
-VkInstance
-VulkanView3D::vkInstance() const
-{
-  gfxApi::Backend* backend = renderlib::graphicsBackend();
-  if (!backend || backend->kind() != gfxApi::BackendKind::Vulkan) {
-    return VK_NULL_HANDLE;
-  }
-  return static_cast<gfxvulkan::Backend*>(backend)->instance();
-}
-
 void
 VulkanView3D::initCameraFromImage(Scene* scene)
 {

agave_app/VulkanView3D.h

@@ -39,8 +39,6 @@ class VulkanView3D
   QSize minimumSizeHint() const override;
   QSize sizeHint() const override;
 
-  VkInstance vkInstance() const;
-
   // VulkanView3D draws to its own native surface (a CAMetalLayer attached to its
   // NSView), so Qt must not paint over it with the regular backing store.
   QPaintEngine* paintEngine() const override { return nullptr; }

agave_app/main.cpp

@@ -227,7 +227,10 @@ main(int argc, char* argv[])
 
     QCommandLineOption listDevicesOption(
       "list_devices",
-      QCoreApplication::translate("main", "Log the known graphics devices (only valid in --server mode)."));
+      QCoreApplication::translate(
+        "main",
+        "Log the known graphics devices and exit. The indices reported are the ones --gpu selects. With the OpenGL "
+        "backend this is only valid in --server mode."));
     parser.addOption(listDevicesOption);
 #if AGAVE_HAS_VULKAN
     const QString defaultGraphicsBackend = "vulkan";
@@ -240,10 +243,14 @@ main(int argc, char* argv[])
       QCoreApplication::translate("main", "backend"),
       defaultGraphicsBackend);
     parser.addOption(graphicsBackendOption);
-    QCommandLineOption selectGpuOption("gpu",
-                                       QCoreApplication::translate("main", "Select GPU/device by index."),
-                                       QCoreApplication::translate("main", "gpu"),
-                                       "0");
+    // No default value here on purpose: parser.isSet() then distinguishes "the
+    // user did not ask for a specific GPU" (auto-select) from "the user asked
+    // for index 0", which must be validated and never silently replaced.
+    QCommandLineOption selectGpuOption(
+      "gpu",
+      QCoreApplication::translate(
+        "main", "Select GPU/device by index, as listed by --list_devices. Defaults to automatic selection."),
+      QCoreApplication::translate("main", "gpu"));
     parser.addOption(selectGpuOption);
     QCommandLineOption serverConfigOption("config",
                                           QCoreApplication::translate("main", "Path to config file."),
@@ -258,7 +265,17 @@ main(int argc, char* argv[])
     bool hasPort = parser.isSet(serverPortOption);
     int port = parser.value(serverPortOption).toInt();
     bool listDevices = parser.isSet(listDevicesOption);
-    int selectedGpu = parser.value(selectGpuOption).toInt();
+    int selectedGpu = gfxApi::kAutoSelectGpu;
+    if (parser.isSet(selectGpuOption)) {
+      bool gpuIndexOk = false;
+      const int requestedGpu = parser.value(selectGpuOption).toInt(&gpuIndexOk);
+      if (!gpuIndexOk || requestedGpu < 0) {
+        LOG_ERROR << "Invalid --gpu value: " << parser.value(selectGpuOption).toStdString()
+                  << ". Expected a zero-based device index; run with --list_devices to see the valid indices.";
+        return 0;
+      }
+      selectedGpu = requestedGpu;
+    }
     gfxApi::BackendKind backendKind =
 #if AGAVE_HAS_VULKAN
       gfxApi::BackendKind::Vulkan;
@@ -315,14 +332,18 @@ main(int argc, char* argv[])
     initParams.headless = isServer;
     initParams.selectedGpu = selectedGpu;
     initParams.windowedContext = bootstrapGLContext.get();
+#ifndef NDEBUG
+    // Enable graphics API validation/debug output in debug builds.
+    initParams.enableDebug = true;
+#endif
 
     if (0 == renderlib::initialize(initParams, listDevices)) {
       renderlib::cleanup();
       return 0;
     }
 
     // Register the cache directory once for the lifetime of the process, after
-    // renderlib has successfully initialized. 
+    // renderlib has successfully initialized.
     // Note that caching stays inert until a CacheConfig enables it.
     CacheManager::initialize(getCacheDirectory());
 

agave_pyvk/src/bindings.cpp

@@ -1,6 +1,7 @@
 #include "renderlib/PythonRenderer.h"
 #include "renderlib/ImageXYZC.h"
 #include "renderlib/VolumeDimensions.h"
+#include "renderlib/gfxapi/Backend.h"
 #include "renderlib/io/ConvertChannelData.h"
 #include "renderlib/io/FileReader.h"
 
@@ -185,7 +186,7 @@ NB_MODULE(_native, m)
     .def(nb::init<const std::string&, const std::string&, int>(),
          nb::arg("mode") = "pathtrace",
          nb::arg("asset_path") = "",
-         nb::arg("gpu") = 0)
+         nb::arg("gpu") = gfxApi::kAutoSelectGpu)
     .def("execute", &execute)
     .def("load_array",
          &loadArray,

docs/agave.rst

@@ -798,11 +798,17 @@ AGAVE supports the following command line options:
 
 ``--list_devices``
 
-  Only valid in server mode on Linux.  AGAVE will dump a list of possible GPU devices and then exit.
+  AGAVE will dump a list of possible GPU devices and then exit. The indices reported here are what ``--gpu`` refers to. With the OpenGL backend this is only valid in server mode on Linux.
 
 ``--gpu number``
 
-  Only valid in server mode on Linux. Selects a device to use from the list provided by list_devices. The device is specified as a zero-based index into the list.
+  Selects a device to use from the list provided by list_devices. The device is specified as a zero-based index into the list. If this option is omitted, AGAVE automatically selects the best compatible device.
+
+  When a device is named explicitly, AGAVE uses exactly that device or fails with an error; it never falls back to a different one. In windowed mode the named device must also be able to present to the application window, which is not true of every GPU on multi-GPU Linux machines. If it cannot, try another index, ``--graphics_backend opengl``, or ``QT_QPA_PLATFORM=xcb``.
+
+``--graphics_backend name``
+
+  Selects the rendering backend: ``vulkan`` (default where available) or ``opengl``.
 
 ``-platform offscreen``
 

renderlib/PythonRenderer.cpp

@@ -99,6 +99,10 @@ PythonRenderer::initialize(const std::string& mode, const std::string& assetPath
     params.assetPath = assetPath;
     params.headless = true;
     params.selectedGpu = selectedGpu;
+#ifndef NDEBUG
+    // Enable graphics API validation/debug output in debug builds.
+    params.enableDebug = true;
+#endif
     if (!renderlib::initialize(params)) {
       throw std::runtime_error("Unable to initialize the headless Vulkan backend");
     }

renderlib/PythonRenderer.h

@@ -49,6 +49,8 @@ class PythonRendererValueError : public std::runtime_error
 class PythonRenderer final : public RendererCommandInterface
 {
 public:
+  // selectedGpu is a zero-based physical device index; gfxApi::kAutoSelectGpu
+  // (-1) lets the backend pick the best graphics-capable device.
   PythonRenderer(const std::string& mode, const std::string& assetPath, int selectedGpu);
   ~PythonRenderer();
 

renderlib/gfxOpenGL/Backend.cpp

@@ -161,6 +161,10 @@ initEGLDisplay(int selectedGpu)
       }
 #endif
     }
+    if (selectedGpu == gfxApi::kAutoSelectGpu) {
+      // No specific GPU was requested; let EGL pick.
+      return getEGLDefaultDisplay();
+    }
     if (selectedGpu >= numberDevices || selectedGpu < 0) {
       LOG_WARNING << "Invalid GPU " << selectedGpu << " requested. Using default gpu.";
       return getEGLDefaultDisplay();