Skip to content

[Bugfix] Skip host_ip/host_port validation when a custom url is given - #614

Open
lsyx6399 wants to merge 1 commit into
AISBench:masterfrom
lsyx6399:fix/model-cfg-url-skips-host-validation
Open

lsyx6399 wants to merge 1 commit into
AISBench:masterfrom
lsyx6399:fix/model-cfg-url-skips-host-validation

Conversation

@lsyx6399

Copy link
Copy Markdown

PR Type / PR类型

  • Bugfix(Bug 修复)

Related Issue | 关联 Issue
Fixes #609

🔍 Motivation / 变更动机

配置里显式给出 url 时,BaseAPIModel._get_base_url() 会直接使用该 URL,并明确在日志中说明
host_ip / host_port 将被忽略:

clean_url = self.url.strip() if isinstance(self.url, str) else ""
if clean_url:
    self.logger.info(f"Using custom URL: [{clean_url}], [host_ip: {self.host_ip}] and [host_port: {self.host_port}]
will be ignored")

但 _validate_model_cfg() 在此之前仍会校验这两个字段,导致用户按常规写法配置
url="..." + host_ip="" + host_port="" 时直接构建失败:

model config validate failed. VLLMCustomAPIChat build failed with the following errors:
{'host_ip': 'host_ip must be a valid IPv4 or IPv6 address',
 'host_port': 'host_port must be a valid port number in the range (0, 65536)'}

用户必须随便填一个 host_ip="9.9.9.9"、host_port=12345 才能跑通(见 issue #609)。

📝 Modification / 修改内容

ais_bench/benchmark/utils/config/build.py::_validate_model_cfg()

  • 遍历字段前计算 has_custom_url = isinstance(url, str) and bool(url.strip());
  • 存在非空 url 时跳过 host_ip / host_port 校验(二者运行时本就被忽略);
  • 其余情况行为不变:无 url 或 url 为纯空白时仍按原规则校验。

不改动模型类,不改变 _get_base_url() 的优先级语义。

📐 Associated Test Results / 关联测试结果

新增 tests/UT/utils/config/test_build.py::test_build_model_from_cfg_with_url_skips_host_validation,覆盖:

  1. url 非空 + 空 host_ip/host_port → 构建成功(修复前抛 AISBenchConfigError);
  2. 无 url + 空 host 字段 → 仍报错;
  3. url=" " → 仍报错(与 _get_base_url() 的 strip 行为一致)。
$ python -m pytest tests/UT/utils/config/test_build.py -q
7 passed in 0.53s

⚠️ BC-breaking (Optional) / 向后不兼容变更(可选)

无。仅放宽"已提供 url 时的无效 host 字段"这一条校验。

⚠️ Performance degradation (Optional) / 性能下降(可选)

无。

@github-actions github-actions Bot added bugfix no_review PR 未包含任何检视意见,合入前必须包含检视意见 labels Oct 28, 2026
@github-actions

Copy link
Copy Markdown

🚫 PR 合入质量检查未通过,PR 已被禁止合入。

未通过项:no_review

  • 无检视意见:请至少邀请一位 reviewer 提交检视意见(Approve / Request changes / Comment 均可)。

请按上述提示整改后,重新推送(push)或评论本 PR 即可触发再次检查。

@lsyx6399

Copy link
Copy Markdown
Author

Hi @Libotry @SJTUyh, could you please take a look when you have time?

This PR fixes #609: when a custom url is provided, BaseAPIModel._get_base_url() already
ignores host_ip/host_port, but _validate_model_cfg() still rejected empty values and
made the config fail to build. The change only skips those two validators when url is
non-empty, plus one UT covering three combinations (with url / without url / whitespace-only url).

pytest tests/UT/utils/config/test_build.py -q → 7 passed.

The "no_review" gate should clear once a review is submitted. Thanks!

@lsyx6399

Copy link
Copy Markdown
Author

Hi @Libotry @SJTUyh, could you take a look when you have time?

This fixes #609. When a custom url is given, BaseAPIModel._get_base_url() already ignores
host_ip/host_port, but _validate_model_cfg() still rejected empty values, so the config
failed to build. The change only skips those two validators when url is non-empty, plus a UT
covering with-url / without-url / whitespace-only-url.

pytest tests/UT/utils/config/test_build.py -q → 7 passed.

Note: as an external contributor I can't request reviewers from the UI — could a maintainer
pick this up? The no_review gate should clear once a review (Approve / Request changes /
Comment) is submitted. Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix no_review PR 未包含任何检视意见,合入前必须包含检视意见

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[易用性] 测试url时没有host_ip和host_port也必须要填

1 participant