I have several working QoL improvements and bugfixes that I'd like to upstream. Per the contributing guide, I wanted to open an issue first. Here's a list of several small, separate items that I'd like to submit a patch for:
- Wheels built with the current
pyproject.toml don't include any packages under src. The current dockerfile only works because it clones the entire source tree to /app and works out of that directory, so runtime imports of src.* packages import files from the source tree rather than the built wheel.
- The config file must live at
project_root / "openarc_config.json". This makes it somewhat difficult to place the config file on another volume, or change between config files. I have a patch that loads the config file path from an env var OPENARC_CONFIG_FILE, falling back to the current path if the var is unset.
- The config file is always written to upon startup, even when the existing config file contents are identical to the content to be written. This makes it impossible for the config file to exist on a read-only volume. This + the above hard-coded config file path limitation means that the application directory must be writable. I have a ~8 line patch that compares new config with the on-disk contents and short-circuits if they are identical.
- Models currently must be referenced in the config file via an absolute path, rather than a path relative to the config file. This makes writing config files ahead of time brittle, as the runtime path to the model must be hard-coded in the config file. I have a patch that allows loading models from a path relative to the config file allowing for config files like
{"model_path": "./models-subdirectory/my-model"} instead of {"model_path": "/model/runtime/mount/path/models-subdirectory/my-model"}.
- The Qwen3 ASR model re quires callers of the Whisper endpoint to always specify an
openarc_asr field, which breaks the API contract and compatibility with other tools. I have a patch that falls back to default ASR config when this is request body field is unset, preserving the contract.
- Compiled Qwen3 ASR models are not cached, causing them to be recompiled on every process restart, even on the same hardware and software stack. Compilation makes several copies of the model in memory, resulting in a huge increase in the peak memory required by the process; around 4GB above what is required post-model load. By simply setting the
CACHE_DIR property on ov.Core() to a path specified by OPENARC_OV_CACHE_DIR, the peak memory requirement is reduced 3x after first startup. This allows for compiling the model once with these higher memory limits, then using lower memory limits for all future model load events (including across process restarts).
- The server always attempts to write to
app-directory/openarc.log. As with the config file saving issue, this requires that the application directory be writable, and writes logs to disk that users may not want. I have a patch that makes this configurable via env var, so that users can configure the server to write to /dev/null, /dev/std{out,err}, or another path of their choosing.
- After d7cb171 was merged,
openarc serve start fails because the locked optimum version references a now-unavailable pytorch field and crashes. I have patch that bumps the lockfile to optimum 2. This works for my use case, but might need other people to test.
Let me know if maintainers would accept some (hopefully all) of these patches. Most are just a couple of lines, with one or two reaching about a dozen lines. If it would be less toil for maintainers, I can combine these into a single ~100 line PR, or I can open separate PRs.
I have several working QoL improvements and bugfixes that I'd like to upstream. Per the contributing guide, I wanted to open an issue first. Here's a list of several small, separate items that I'd like to submit a patch for:
pyproject.tomldon't include any packages undersrc. The current dockerfile only works because it clones the entire source tree to/appand works out of that directory, so runtime imports ofsrc.*packages import files from the source tree rather than the built wheel.project_root / "openarc_config.json". This makes it somewhat difficult to place the config file on another volume, or change between config files. I have a patch that loads the config file path from an env varOPENARC_CONFIG_FILE, falling back to the current path if the var is unset.{"model_path": "./models-subdirectory/my-model"}instead of{"model_path": "/model/runtime/mount/path/models-subdirectory/my-model"}.openarc_asrfield, which breaks the API contract and compatibility with other tools. I have a patch that falls back to default ASR config when this is request body field is unset, preserving the contract.CACHE_DIRproperty onov.Core()to a path specified byOPENARC_OV_CACHE_DIR, the peak memory requirement is reduced 3x after first startup. This allows for compiling the model once with these higher memory limits, then using lower memory limits for all future model load events (including across process restarts).app-directory/openarc.log. As with the config file saving issue, this requires that the application directory be writable, and writes logs to disk that users may not want. I have a patch that makes this configurable via env var, so that users can configure the server to write to/dev/null,/dev/std{out,err}, or another path of their choosing.openarc serve startfails because the locked optimum version references a now-unavailable pytorch field and crashes. I have patch that bumps the lockfile to optimum 2. This works for my use case, but might need other people to test.Let me know if maintainers would accept some (hopefully all) of these patches. Most are just a couple of lines, with one or two reaching about a dozen lines. If it would be less toil for maintainers, I can combine these into a single ~100 line PR, or I can open separate PRs.